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PREFACE TO THE AMERICAN EDITION 


THIS BOOKLET is a translation of Part Three of Mathematical Conver- 
sations by E. B. Dynkin and V. A. Uspenskii, published as Number 
6 in the Russian series, Library of the Mathematics Circle. The book 
is based on the material covered during the academic years 1945-46 
and 1946-47 in one of the sections of the School Mathematics Circle 
at Moscow State University. One of the authors was the instructor 
of the section, and the other was a participant. 

The primary aim was not so much to impart new information as 
to teach an active, creative attitude toward mathematics. The most 
successful topics took shape only as the work progressed. A series of 
consecutive meetings was devoted to each topic. A meeting would 
usually begin with problems whose formulation required no new 
concepts, but whose solution led the students directly into a new 
area of inquiry. These problems would sometimes be solved during 
the meeting, but more often they were left as homework. At the next 
meeting, the instructor would discuss the solutions of the problems 
and then use them as a basis for generalizations. Whenever possible, 
the material was presented in sequences of related problems. 

This booklet presents one of the topics, Random Walks, in a con- 
siderably revised and expanded form. Like the discussions on which 
it is based, it retains the practice of interrupting the presentation 
with problems whose solutions are essential to what follows. To 
understand this material, the reader should be familiar with high 
school algebra. 


INSTRUCTIONS FOR THE USE OF THIS BOOKLET 


This booklet is devoted to a single topic, and it should therefore 
be read in order. Moreover, it is designed for the reader’s active 
participation, and the problems form an organic part of the text. 
Most of the problems are grouped in sequences, each sequence form- 
ing a unit and building up to a final result contained in the last 
problem of the sequence. Sometimes the aim of a sequence of prob- 
lems is not some definite result, but rather mastery of a new method. 


Finally, a few of the problems are practice exercises, designed to help 
the reader master new concepts (for example, Problems 1-3). 

Before attempting to solve a problem, the reader should examine 
all the problems in the given sequence. Solutions are provided fol- 
lowing the Concluding Remarks, but it is recommended that the 
reader look at them only after he has tried to solve all the problems 
of a sequence. If he looks at the solutions too soon, they may set his 
mind working in a certain direction, but with independent thought 
he may arrive at new and original methods. The experience of the 
School Mathematics Circles has shown that sometimes simpler and 
more elegant solutions are found than those expected by the authors 
of the problems. 

The reader may not always be able to solve all the problems of a 
sequence independently. If, after solving the first few problems, he 
should run into difficulties, he may find it helpful to read the solu- 
tions of the problems he has already solved. If these do not suggest 
an approach to the next problem, he should look at its solution, and 
then proceed to try the rest by himself. Eventually, he should read 
all the solutions, whether or not he has succeeded in solving the prob- 
lems independently, as they have been carefully prepared, and many 
of them are accompanied by conclusions and remarks of a funda- 
mental nature. 

Although the problems here are basic, this is by no means merely 
an exercise book. The text is also important. The relation between 
problems and text differs in the various chapters. In some the 
essential ideas are set forth in the text, but in others they are in the 
problems, and the text merely introduces concepts and states results. 
The text and problems are always closely related and must be read 
in the order in which they appear in the book. 

In conclusion, we advise the reader not to begrudge the time 
spent on solving the problems. Each sequence, indeed each problem, 
solved independently enlarges the arsenal of resources at his disposal. 
One idea arrived at independently is worth a dozen borrowed ones. 
Even if persistent attempts to solve a problem do not lead to success, 
the time is not spent in vain, as he will then see its solution in a new 
light. He can look for the reason for his failure and can discover the 
fundamental idea that leads to success. 
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Random Walks 


Introduction 


There is a well-known board game called Circus. Two players 
take turns rolling a die, and each in turn moves his piece forward 
on a square board which is divided into 100 numbered squares. 
The piece is moved as many squares forward as the spots on the 
die indicate. At the beginning of the game, both pieces are placed 
on the first square. The winner is the one who first reaches the 
square numbered 100. There is one further rule: If a piece reaches 
a square with a red number, it moves to another square (either 
forward or backward) whose number is blue; this move is specified 
by the “circus act”’ marked on the board. Obviously, in this game 
the motion of the pieces depends not on the skill of the players, 
but rather on chance (assuming the die is rolled “fairly”). The 
motion of these pieces is one simple example of a random walk. 

We give another example of a random walk. Two friends live in 
a city whose map is shown in Figure 1. They leave their house, 


ZAZA. 
20a. 
ZA007 
Z0C0Z 
TIAA 


Fig. 1 


which stands at the intersection A, and set out to go for a walk. 
However, they disagree as to the route they are to take; they agree 
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that at each intersection, beginning with A, they will toss two coins 
and proceed north, east, south, or west, depending on which of the 
four possible tosses (heads-heads, heads-tails, tails-heads, tails-tails) 
comes up. Thus, they toss the’ coins and begin their walk at the 
intersection A; when they reach the following intersection, they 
again toss the coins and choose their further path according to the 
result. If they come to the edge of town (for example, to the point 
X), they will turn around and come back. 

Cases similar to the examples just given (but much more compli- 
cated) are encountered in nature. Brownian motion will serve as an 
example: “If a light powder is suspended in water, and if a drop of 
this water, together with the particles contained therein, is placed 
under the microscope, one can observe that the particles appear to 
be alive, because they are in continuous zigzag motion.” Here, the 
random path (Fig. 2) has an essentially more complicated character 
than in the previous example: in the first 
place, the particles can change the direction 
of their motion at any moment, while in the 
example of the walk in the city, this was 
possible only at intersections; in the second og 
place, the previous example permitted only 
four possible directions of motion, whereas Fig. 2 
the particles can move in any direction. 

In this section, we shall consider a few simple examples of 
random walks. We returm to the board game described above and 
investigate the duration of the game. To solve this problem we con- 
sider a less complicated board (Fig. 3), one that has only 25 squares, 
and add the rule that a piece landing 
on square 24 must automatically return 
to the starting point. There are no other 
rules for moving from square to square, 
and the squares are all colored with 
the same color. How long does it take 
to play a complete game on this board? 
It is possible for a player to reach 
square 25 as early as the fourth move; 
this occurs if he rolls four successive 
sixes. On the other hand, if each play- 
er’s piece repeatedly lands on square 





24, the game will not have ended after 1,000 rolls of the die. It can 
even happen that the game never ends at all; this occurs, for 
example, if both players roll the following sequence: 


6, 6, 6, 5, 6, 6, 6, 5, 6, 6, 6, 5, 6, 6, 6, 5, 6, 6, 6,5,.... 


In this case, no bound can be given for the duration of the game. 
But try playing one game after another. You will be convinced 
that the game does end and, in fact, rather quickly. Just what is the 
trouble here? 

We have already determined that it is impossible to give a defi- 
nite bound for the duration of the game. The situation is basically 
altered, however, when an absolutely certain answer is not required. 
To clarify this last remark, let us consider some more examples. 


EXAMPLE 1. Suppose we have an urn which contains 1,000 balls; 
suppose, also, that one of the balls is black, the others white. A ball 
is taken from this urn at random. Can it happen that the black ball 
is chosen? 

Answer. It is certainly possible, but it is not at all likely. 


EXAMPLE 2. Suppose a man who does not know Russian starts to 
type on a Russian typewriter. Is it possible that he thereby writes 
Pushkin’s story, The Captain’s Daughter? 

Answer. Obviously, this is not absolutely impossible. However, 
it hardly appears to be a real possibility to anyone. (Indeed, the 
essays of two school children may also coincide word for word. 
But, as a rule, one quite correctly regards this as evidence that one 
of the essays was copied.) 


EXAMPLE 3. According to the kinetic theory of gases, air is com- 
posed of a great number of molecules that are moving randomly. 
There is almost no interaction between the individual molecules; 
hence, the position of one molecule in space does not influence the 
positions of the other molecules. We imagine the space in the room 
we are in to be divided into an upper and a lower half. It is not 
impossible (this follows from the kinetic theory of gases) for all of 
the molecules of air in the room to go into the upper half, suffo- 
cating everyone in the room. This conclusion appears far-fetched 
to us. But does this mean that the kinetic theory is false? 


Answer. It is to be hoped that the reader does not draw this 
conclusion, in view of the examples discussed above. The phenom- 
enon that we have just presented is indeed not absolutely impos- 
sible; one can, however, view it as practically impossible. The basis 
for this conclusion is even more apparent in this example than in 
the previous example: there are incomparably more molecules in a 
room than there are letters in the story The Captain’s Daughter. 


Thus, we may quite correctly regard a highly unlikely occurrence 
as impossible for all practical purposes. Moreover, there are differ- 
ent degrees of unlikelihood. The unlikelihood of the occurrence of 
the event in the second example is much greater than that in the 
first, and the unlikelihood in the third example is much greater 
than that in the second. 

We return again to our problem, which we now reformulate as 
follows: What /imit can be estimated for the length of the game, so 
that a game whose length exceeds the estimated limit is practically 
impossible in the same degree as are the events of our examples? 

How one solves this and similar problems will become clear to 
the reader after working through this book. We shall give only the 
answer here. If we estimate that the game ends after at most 200 
throws, an error will be about as unlikely as the event in the first 
example. If the estimate is increased to 30 million, an error will be 
about as unlikely as the event in the second example. If the esti- 
mate is raised to 1075, an error will be as unlikely as the event in 
the third example. 


1. Probability 


1. FUNDAMENTAL PROPERTIES OF PROBABILITY 


First we shall learn how to calculate probabilities. Consider two 
urns with 100 balls in each. In the first urn, one ball is white, the 
other 99 black. In the second urn, 10 balls are white and 90 black. 
From which urn is one more likely to draw a white ball?! The 
reader will answer without hesitation: from the second. If we ask 
how many times greater this probability is, the reader will certainly 
answer that it is ten times as great. Suppose, now, that we have a 
third urn, in which all 100 balls are white. As before, we conclude 
that a white ball can be drawn from the third urn with 100 times 
the probability as from the first urn. The ball taken from the third 
urn will surely be a white ball. If we define the probability of this 
last, inevitable event to be the number 1, it follows from what has 
been said that the probability of drawing a white ball from the first 
urn is equal to $5, and the probability of drawing a white ball 
from the second urn is 7%. 

We consider the general case, in which the urn contains n balls, 
of which m are white. From the same considerations as above, we 
can conclude that the probability of a white ball being drawn from 


the urn is —“. The urn problem is extraordinarily useful because 
n 


many problems can be reduced to this form. 


EXAMPLE 1. What is the probability that heads will come up on the 
toss of a coin? We consider an urn with one white and one black 
ball. Let the white ball correspond to heads, the black to tails. 

Answer. Obviously, the probability that heads will come up is 
equal to the probability that one draws a white ball from our urn, 
and this equals 5. 


1 Here it is assumed that the balls in the urns are completely uniform and well mixed 
and that one does not look when drawing; then, one has the same probability of 
drawing any ball. 


EXAMPLE 2. What is the probability that one rolls a five with a die? 
The problem may be thought of as an urn with six balls, one 
of which is white. What is the probability that the white ball will 
be drawn? » 

Answer. 4. 


EXAMPLE 3. A domino is drawn at random from a box of dominos. 
What is the probability that there is a six on one end of this 
domino? We consider an urn with 28 balls, of which 7 (those corre- 
sponding to the 7 dominos that have a six on one of their halves) 
are white. 

Answer. The probability that a domino of the desired kind is 
drawn is equal to the probability that a white ball is drawn from 
the urn, that is, 5. 


EXAMPLE 4. There are 5 red, 7 blue, and 13 black balls in an urn. 
What is the probability that either a red or a blue ball is drawn? 
There are 12 white and 13 black balls in a second urn. What is the 
probability that a white ball is drawn? 

Answer. The probability that either a red or a blue ball is drawn 
from the first urn is equal to the probability that a white ball is 
drawn from the second urn, that is, 42. 


In general, a trial may have n equally probable outcomes, of 
which m yield a desired event A, the others yielding an undesired 
event. Each such trial is equivalent to drawing a ball from an um 
containing n balls, of which m are white and the rest black. The 
occurrence of the event A has exactly the same probability as the 
drawing of a white ball from the urn, that is, 


m 


nn’ 


DEFINITION. The probability of an event A is equal to the num- 
ber of possible favorable outcomes divided by the total number of 
Possible outcomes. 


We denote the probability of the event A by 
P(A). 


We now formulate the following properties of probability: 


Property 1. If the event A implies the event B, that is, if each 
occurrence of the event A is followed by an occurrence of event B 
(or, the event B always occurs when the event 4 does), then 


P(A) < P(B). 


Property 2. If the events A and B are mutually exclusive (that is, 
it is impossible that both A and B occur), then 


P(A + B) = P(A) + P(B), (1) 


where A + B is understood to mean the event that consists of the 
occurrence of either A or B. 

Property 3. If the events A and B are exact opposites of each 
other, that is, if the occurrence of A is the same as the nonoccur- 
rence of B, then 


P(A) + P(B) = 1. (2) 
Property 4. If the event E is certain, that is, if E must occur, then 
PE) = 1, 


Property 5. If the event O is impossible, that is, if O cannot 
occur, then 


P(O) = 0. 


It is easy to obtain these properties from a consideraton of the 
urn problem, and the reader may derive them for himself.1 We shall 
only clarify a few definitions: an example of mutually exclusive 
events is the drawing of a blue ball and the drawing of a red ball 
(when only one ball can be drawn from the urn); an example of 
opposite results is the tossing of a coin, where either heads or tails 
must come up. The drawing of a white ball from an urn that con- 
tains only white balls is a certain event; the drawing of a black ball 
from this urn would then be an impossible event. 

Although many problems can be reduced to the urn problem, 
there are many (and among them the most interesting) that cannot 
be reduced to this problem. However, Properties 1-5 of probability 
are always true. 


1We have already derived a special case of Property 2 in Example 4 on page 6. 
Here we restate the Example: A is the event of a red ball being drawn; B is the event 
of a blue ball being drawn; and A + B becomes the event of either a red or a blue 
ball being drawn. 


2. CONDITIONAL PROBABILITY 


We now wish to become acquainted with so-called conditional 
probability, first, a few examples. 

At recess, students of the first and second grades gather in the 
playground to play. Eleven pupils of the first grade take part, 
8 boys and 3 girls, together with 6 pupils of the second grade, 
2 boys and 4 girls. It is decided by lot who is to begin.1 What is the 
probability that the lot falls on a first-grade student? To calculate 
this probability, the number of students of the first grade must be 
divided by the number of all participants. The result equals 44. 

We now assume that we know that the game will be started by 
a boy, and ask what influence this has on the probability of inter- 
est to us. We want to know what the probability is now for a 
member of the first grade to start the game. All 10 boys that par- 
ticipate in the game are equally likely to begin. Of these 10 boys, 
8 are pupils in the first grade. Hence, the probability that a mem- 
ber of the first grade begins is, in this case, equal to 7%. We see that 
the probability has changed. 

We have obtained the conditional probability that a member of 
the first grade begins, on condition that the game is begun by a boy. 


DEFINITION. The conditional probability of an event B given an 
event A is the probability that the event B will occur, if it is known 
that a previous event A is certain to occur. It is denoted by 


P(BIA). 


Problem 1. In the example above calculate the conditional proba- 
bility that a member of the first grade begins, on condition that the 
game is begun by a girl. 


In our example, let B denote “the game is begun by a member 
of the first grade” and A denote “a boy begins.” As we calculated, 
P(B) = ++ and P(B\A) = &. Hence, in this case P(B|A) ~ P(B). 
The occurrence of the event A thus has a significant influence on 
the probability of the event B. 

We add yet another property of probability to the five already 
enumerated in section |. 

‘The drawing must guarantee that the probability of any participant beginning the 


game is the same. Among the possible forms of the drawing, that one is best which 
achieves this equality of probability most completely. 
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Property 6. 
P({AB) = P(A) P(BIA). (3) 


By AB, we are to understand the eventin which both A and B occur. 


We shall now indicate how one can derive this property. We 
shall verify Property 6 by means of our previous example of the 
students in the playground. The person who begins the game is 
determined by lot. As before, let A be the event “A boy begins” 
and B the event “A member of the first grade begins.” Then, AB is 
the event “A boy of the first grade begins.” Of the 17 possible out- 
comes, the event AB can occur in 8 ways (8 boys are pupils in the 
first grade); the event A can occur in 10 ways. Hence, P(AB) = & 
and P(A) = 49. We have already found that P(B|A) = 3%; thus we 
have 


P(AB) = P(A) P(BIA). 


If A and B are independent events, neither the occurrence nor 
nonoccurrence of the event A has any effect on the probability of 
the event B; hence, the conditional probability P(B|A) of the event 
B on the condition A is equal to the unconditional probability P(B): 


P(B|A) = P(d). 
In this case, formula (3) takes the form 
P(AB) = P(A) P(B), 
and we obtain: 
Property 6a. If A and B are independent events, then 
P(AB) = P(A) P(S). (4) 


EXAMPLE. Successive flips of a coin are independent events. Hence, 
the probability of obtaining heads twice in a row is P(“heads come 
up on the first toss” and “heads come up on the second toss”) = 
P(“heads come up on the first toss”) + P(“heads come up on the 
second toss”) = 4°4 = §. 


Formulas (1) and (4), the formulas for addition and multiplica- 
tion of probabilities, can easily be generalized to the case of more 
than two mutually exclusive or independent events. Let n events 
A, Ag,..., An, any two of whith are mutually exclusive, be given. 
Then ; 


P(4; + Ao + Az +--+ +An) 
= P(A1) + P(A2) + P(A3) + --- + P(An). 5) 


We prove this formula for n = 3. The event A3 is mutually 
exclusive of both A, and Ag; from this it clearly follows that A3 and 
A, + Ag are mutually exclusive. Then, by Property 2, we have the 
following formula 


P(A, + Ao + A3) = P(A, + Aa) + P(A3). (5’) 
But A, and Ag are also mutually exclusive, so that 
P(A; + Ao) = P(A) + P(A2). (5”’) 


Formula (5) follows, for the case n = 3, from (5’) and (5”). Formula 
(5) can be proved analogously for arbitrary n. 

If A; and Ag are independent, and if A,A2 and Az are also 
independent, then 


P(A142A3) = P(A142) P(A) = P(A1) P(A2) P(A). 


More generally, if 4, and A2 are independent, and, also, each of the 
pairs of events A,A2 and A3, A1A2A3 and Ag, ..., A1A2A3--+ An-1 
and A, are independent, then 


P(A140A3 - + An) = P(A1) P(A2) P(A) --- P(Az). (6) 


For example, let A; be the event that the kth toss of a coin is 
heads. Then, the probability that the coin comes up heads a times 
is given by formula (6). 

P(41A2--- An) = P(A1)P(42)--- P(An) = 4-40-54 = ore 

n times 
This same number gives the probability that the coin comes up 


tails n times and, in general, that on nv tosses any previously speci- 
fied sequence of heads and tails comes up. 


Problem 2. What is the probability that no six will come up on six 
rolls of a die? 
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Problem 3. We understand by p the probability that the target is hit 
on one shot. Calculate the probability that in n shots, one hits the 
target. : 


We mentioned in the introduction that events of small proba- 
bility can, with good grounds, be regarded as practically impossible, 
and that there are varying degrees of unlikelihood. We shall make 
this more precise. 

If one is to apply the methods of probability theory to the study 
of a phenomenon in nature, he must each time choose an arbitrarily 
small number e, the permissible probability of a deviation (an error). 
If we have predicted a course of events by arguments based on 
probability theory, we must admit the possibility of error in our 
prediction and demand only that the probability of this error is not 
greater than e. We start with the assumption that all events whose 
probability is smaller than e are to be regarded as practically impos- 
sible, and that all events whose probability is greater than | — e€ 
can be assumed to be practically certain. 


DEFINITION. The number e is the permissible degree (or the mag- 
nitude) of uncertainty and the number | — « is the required degree 
(or magnitude) of certainty. 


Obviously, the value of e« must be chosen for each individual 
problem according to the practical requirements on the correctness 
of the conclusions. Frequently used values of e are 0.01, 0.005, 
0.001, and 0.0001. 

Let us clarify our definition and discussion by considering the 
example of the number of shots that are necessary for a single hit 
on a target (see Problem 3 above). Suppose that for each shot the 
probability of a hit is 0.2. How many shots must be made to hit 
the target once? It is clear that the number of shots cannot be 
given with absolute certainty. For it can happen that the target is 
hit on the first shot; at the same time, one cannot exclude the pos- 
sibility that after 100 or 200 shots none have yet hit the target. 
Hence, we shall not seek after absolute certainty, but rather intro- 
duce a permissible degree of uncertainty e. It appears entirely accept- 
able, for example, to set the value ¢ equal to 0.001. The statement 
that the target will have been hit after 1 shots is false with proba- 
bility (1 — 0.2)" = (0.8)". 


We now choose n so that 
(0.8)" < 0.001. 


The smallest value of n that satisfies this inequality is 31. (It is easy 
to calculate this with the aid of a table of logarithms.) Hence, the 
statement that the target will be hit at least once after 31 shots 
is false with a probability that does not exceed the permissible 
bound 0.001. Hence, under our requirements for the degree of 
certainty, we can say that it is practically certain that the target will 
be hit after 31 shots. 

The number n varies with the required degree of certainty e. 
The values of n for different degrees of certainty are compiled in 
the following table: 





EXAMPLE. Calculate the probability that no six comes up on infi- 
nitely many rolls of a die. 

SOLUTION. We first calculate the probability of the event B, 
that no six has come up after n rolls. In the solution of Problem 2, 


page 10, we found P(Bs) = (2)’. Analogously, we find (by for- 
mula (6)), 


PB.) = (2)’, 


for arbitrary n. We denote by B the event of interest to us, that no 
S1X appears in an infinite sequence of rolls. If the event B occurs, 
then all of the events B,, Bo, ..., Bn, ... must occur. Hence, on the 
basis of the first property of probability: 


P(B) < P(B1) = 2, 


P(B) < PBs) =(2), 


Sy 


The numbers =, (2)’, ree (2). ... are the terms of an infinite 
decreasing geometric progression; these terms will eventually be 
smaller than any predetermined positive number, provided n is 
sufficiently large.1 Hence, P(B) also becomes smaller than any posi- 
tive number, that is, P(B) = 0. Hence, the probability that no six 
appears in an infinite sequence of rolls of a die is equal to zero. 


The events that we have dealt with up to this time have either 
been impossible, or, if possible, had a probability greater than zero. 
Here we encounter for the first time an event whose probability is 
equal to zero and which, nevertheless, appears to be logically pos- 
sible. We could not obtain this result if the probability of our event 
were Calculated by the rule set forth at the beginning of this section, 
namely, as the ratio of the number of favorable events to the total 
number of all possible events. 

The result that we have obtained as our solution to this exercise 
can be interpreted in the following way. However high we may 
wish the degree of certainty to be, a number of rolls can be given 
for which the six must come up at least once with this certainty.” 
This is the precise meaning of the statement that the six comes up 
with a certainty of | in infinitely many rolls. 


3. THE FORMULA FOR COMPLETE PROBABILITY 


DEFINITION. A system of events Ay, A2, A3, ..., An is called 
complete if at least one of the events must occur (in other words, 
if the event Ay + Az + Az +--+ + An is certain). 


If the events Ay, Ao, ..., An form a complete system and if they 
are mutually exclusive, then 
P(A;) + P(A2) + - >> + P(A,) = 1 (7) 


(this follows from formula (5) and Property 4). 


1 Proofs of this are found in the section on limits in many calculus textbooks. 


2{In fact, if we say that a six occurs in 7 rolls, we can err with a probability of (3)". For 
sufficiently great n, the probability of an error can be made arbitrarily small. 


13 


Property 7. (The formula for complete probability.) Let a com- 
plete system of mutually exclusive events Aj, Ae, A3,..., An be 
given. Then the probability of an arbitrary event B can be calcu- 
lated by the formula > 


‘ 


P(B) = P(Ax)P(BIAs) + P(d2)P(BlA2) + «+: 
+ P(A,)P(BIA,). (8) 


To prove this, note that since one of the events Ai, A2,..., An 
occurs, the occurrence of the event B is equivalent to the appear- 
ance of one of the events BA;, BA», ..., BAn. Hence, 

P(B) = P((A1B) + (42B) +--+ + (AnB)). 
Since the events 4;B, AoB,..., AnB are mutually exclusive (as Aj, 
Ag,..., Ay are mutually exclusive), we have, by formula (5), 


P(B) = P(A1B) + P(4A2B) + --- + P(A7B). 
Applying Property 6, we obtain 
P(B) = P(A1) P(BIA1) + P(Az) P(BIA2) + - - - + P(An) POBIAn). 
Let us use this formula to calculate the probability that in the 
game we described on page 8 a member of the first grade begins. 


Here, B means that a member of the first grade begins, A; that a 
boy begins, and A2 that a girl begins. We obtain 


10 ei 8 3 
P(A) = 7 P(A2) = 7 P(BIA,) = T0° P(B\A2) = F 


P(B) = P(A1) P(BIA1) + P(A2) P(B|A2) 
HON Be 9 Se 


=a 10 IT 7 tT 


This result agrees with the previous calculation. 


Problem 4. Two players alternately toss a coin, and the one that first 
tosses heads wins. What is the probability that the game never ends? 
What is the probability that the first player wins? What is the prob- 
ability that the other player wins? 


Problem 5. A particle at point A (Fig. 4) can, in the next unit of 
time, remain at A with the probability p31, or move to point B with 
the probability pie. If it is at B, it can, in the next unit of time, 
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remain at B with the probability pee or Pi2 
move to point A with the probability po. rm OO) P22 


What is the probability that the particle a raul 
iS at point A after n units of time, if at Fig. 4 
the initial moment it is: 

(a) At point A? (b) At point B? 


Many different phenomena can be reduced to the diagram in 
Figure 4. Following the example of the distinguished Russian 
mathematician A. A. Markov, we consider the first 20,000 letters of 
the poem Eugene Onegin (except b and b). Our particle is at point 
A if the letter is a vowel and at point B if the letter is a consonant. 
The succession of vowels and consonants is represented by the 
motion of the particle on the diagram shown in Figure 4. Obviously, 
the probability that the letter following a vowel is a consonant is 
greater than that the letter is once more a vowel. In fact, Markov’s 
calculations show that the probability pie for the appearance of a 
consonant under the condition that the previous letter was a vowel 
is approximately equal to 0.872, while the probability pi; for the 
appearance of a vowel under the same condition is equal to about 
0.128. It was shown in the same way that po: ~ 0.663 and 
p22 0.337. 

Similar counts made of the first 100,000 letters of Aksakov’s 
story The Childhood of Bagrov’s Grandson give a somewhat different 
result: 


pir 0.147; por ~ 0.695; 
Piz = 0.853; P22 = 0.305. 


With certain limitations, a number of meteorological phenomena 
can also be handled in a similar fashion, for example, the sequence 
of clear and cloudy days. The probability that a cloudy day will 
follow a cloudy day is greater than the probability that the following 
day will be clear; the probability that a clear day will follow a clear 
day is greater than the probability that the following day will be 
cloudy. The probability of a change from a clear day to a cloudy 
one and from a cloudy to a clear one, etc. (a movement of the 
particle in Problem 5 corresponds to this change), proves to be 
approximately constant for a definite place and season and can be 
calculated from observations. 


Problem 6. Two identical-looking urns stand in a~room. Suppose 
there are a balls in the left urn and b in the right. Several people 
come into the room, one after’ the other, and either transfer a ball 
from the right urn to the left or i from the left urn to the right. It is 
assumed that the probability that a ball is transferred from the 
right urn to the left is equal to the probability that a ball is trans- 
ferred from the left urn to the right, that is, 4. The experiment goes 
on until one of the urns is empty. What is the probability that the 
left urn becomes empty? What is the probability that the right urn 
becomes empty? What is the probability that the experiment does 
not end? 


Problem 7. A caterpillar crawls along the 2 B 
edges of a wire cube (Fig. 5). On reaching a JA 
corner, the probability that it will crawl onto 1 

any particular edge that leads out from this 

corner is 4. The points A and B are daubed 

with glue. The caterpillar starts out from the 3 4 
point 0. What is the probability that it sticks 

to the point A? What is the probability that 0 5 

it sticks to the point B? Fig. 5 
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2. Problems Concerning 
a Random Walk on 
an Infinite Line 


4. GRAPH OF COIN TOSSES 


We mark the points 0, +1, +2, +3,... ona straight line and 
carry out the following experiment: We place a marker on the 
point O and toss a coin. If heads comes up, we move the marker 
one place to the left, and if tails comes up, we move the marker 
one place to the right. Now we toss the coin for a second time, for 
a third time, etc., and each time move the piece according to the 
result of the toss. We can assume that the two possible outcomes of 
a toss have equal probabilities, so that for each toss the probability 
that the marker is moved to the left is exactly as great as the prob- 
ability that the marker is moved to the right; that is, the probability 
is 4. 

Clearly, after the first toss, the marker is on the point —1 or 
1; after the second toss, on one of the points —2, 0, or 2; after the 
third, on —3, —1, 1, 3, etc. The diagram shown in Figure 6 gives 
a graphical picture of the possible 
positions of the marker at each —4—3-2-1 
moment. ‘ 

The points shown in this dia- 


0 12 3 4 


gram form a triangle. (This triangle . : 

can be continued downwards in- . ‘ ° 
definitely.) The vertex of the tri- : ° . : 
angle lies under the number 0, cor- °° e ° ° ° 
responding to the fact that 0 was Fig. 6 


the initial position of our marker. 

The first row of the triangle consists of two points. The numbers 
lying over these points, —1 and 1, show where the marker can 
stand after the first toss. The following row shows where the marker 
can stand after the second toss, and so on. 


5. THE TRIANGLE OF PROBABILITIES 


The diagram shown in Figure 6 exhibits all of the possible posi- 
tions of the marker, but of these some are more, others less prob- 
able. We seek to calculate these probabilities. At the-beginning, the 
marker is on the point 0 with probability 1. After the coin has 
been tossed for the first time, the marker is found with a probability 
of 4 on each of the points —1 and |. After the second toss, one can 
have obtained the following results: 


heads-heads, heads-tails, tails—heads, tails—tails. 


These four results are all equally probable, and, consequently, each 
has the probability 4. After the first result, the marker is on point 
—2; after the second and third, on 0; and after the fourth, on +2. 
Hence, after the first two tosses the marker is found on the point 
—2 with a probability of 4; on the point 0 with a probability of 3; 
and on the point +2 with a probability of 4. Similarly, the proba- 
bility of each possible position of the marker after the third, fourth, 
... toss can be calculated. If one replaces every point of the dia- 
gram by the corresponding probability, one obtains the triangle of 
numbers shown in Figure 7. This 

triangle (we shall callit the triangle —4—3-—2—-1 0 1 2 3 4 
of probabilities) has a noteworthy 
property: each of its numbers is 


equal to half of the sum of the two 2 2 

numbers standing above it. This 4 4 4 
property can be easily verified for $ 3 3 8 

all of the numbers shown in Fig- ~ * ae uae 
ure 7. It is clear, however, that this Fig. 7 


check still does not prove that this 
property continues to hold for an arbitrary continuation of the 
triangle. 


Problem 8. Prove, with the help of the formula for complete prob- 
ability (Property 7, page 14), that 


Za = 5 (Lut + Zu 1) (1) 


where Z,* denotes the probability that the marker is at the point k 
at time n. 
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With the help of the rule of the half-sum, the triangle of proba- 
bilities can be easily continued. The first nine rows of the triangle 
of probabilities are written out in Figure 8 (not counting the zeroth 


1 Oth row 

$ 4 Ist row 

4 4 4 2nd row 

+ # # ¢ 3rd row 
&birereisé & & 4th row 
be BR SD Sth row 
tt HHH A & 6th row 


qe ovis tes tee tes ts the the 7th row 
she ost OF PK TE ASE 76 Ee Ok Sth row 
sh sh Ae Sh HS HS SA SS sh ot Yth row 


row, the vertex of the triangle). Their direct calculation (by the 
method used for the first four rows) would not be easy. 


Problem 9. Prove that the sum of the elements of each row of the 
triangle of probabilities is equal to one. 


We remark that the rule of the half-sum is equivalent to the fol- 
lowing halving-rule. We consider an arbitrary row of the triangle, 
halve every number of this row, and place one half below and to 
the right, the other half below and to the left (see Figure 9, in 


16 16 6 16 1s 
va ae er to ae \ 


which this is carried out for the fourth row). If we add the numbers 
that stand on the same point, we obtain the next row of the triangle. 

Let us now imagine that at the initial time there is a unit mass 
at the point O, and that in the course of a second it splits into two 
equal parts, one half moving to the left and the other half to the 
right. During the second second these halves again divide into two 
equal parts, one of which moves to the right, the other to the left, etc. 
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It is clear that the sum of the masses that are found at the point k 
after n seconds is equal to the number in our triangle that stands 
in the kth place of the nth row. This connection between the prob- 
lem of the random motion of a marker and the problem of the 
shifting of a dividing mass is very useful in the solution of a num- 
ber of problems.! 

If the first row of the triangle of probabilities is multiplied by 2, 
the second by 22, the third by 23,..., the nth by 2”, we obtain a 
triangle that consists only of whole numbers. The reader can easily 
verify that, in this new triangle, every number is equal to the sum 
of the two numbers standing above it. This triangle is called 
Pascal's triangle; see E. B. Dynkin and V. A. Uspenskii, Problems 
in the Theory of Numbers (Boston: D. C. Heath and Company, 
1963), Chapter 3, section 18. 

Let us divide every element of the triangle of probabilities by 
its left neighbor. Of course, the elements of the left edge of the 
triangle have no left neighbors. We strike out these elements and 
what remains is the quotient triangle in Figure 10. The law by 


i 


fon 
hn 

ten ey 
Nie ho 

wha Nie |r 
who tops 

ah Sao ne 
ENN) he 

tno ae 
uh 

Or 


bh 
nin 
ohn 
alo 
le 
apy 
~e 


which this triangle is constructed is easy to discover. In the nth 
row, the denominators of the fractions run through all the numbers 
from | to n, while the numerators run from n to 1. We leave it to 
the reader to verify this law with the aid of formula (1). 


! An analogous scheme was considered in one of the problems of the Eighth Moscow 
Mathematical Olympiad. This exercise dealt with 2" men who start out from the 
vertex of the triangle in Figure 6, half of them going down and to the left, and half 
going down and to the right. At each point, they continue according to the halving- 
rule, It was asked how many people there are at each point of the nth row. 
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The triangle of probabilities can easily be recovered from the 


quotient triangle. We begin the nth row with the number +. and then 


reconstruct all of the elements of this row, one after the other, by 
multiplying the element already obtained by the corresponding 
element of the nth row of the quotient triangle. We thus obtain the 
triangle of probabilities in the form given in Figure 11. 





4 at 
4 44 ce 
$ tt ee a a a Sa 
16 16° 1 Mes. Getree. Terk s'4 
Pe a ere ter te seer erin 


From this, it can be seen that the (k + 1)st element of the nth 
row of the triangle of probabilities is equal to 


Sle ite ee BD MEI T, (2) 
| 2 3 k 


6. CENTRAL ELEMENTS OF THE TRIANGLE OF PROBABILITIES 


We are particularly interested in the central elements of the tri- 
angle of probabilities, that is, the elements that lie on its axis of 
symmetry. These elements are found only on even rows. The cen- 
tral element Z,°, which lies on the 2kth row, is the probability 
that the marker will have returned to the initial position after 2k 
moves. Denoting the probability of this event by wa., we have 


o-= i}; ae fe re | > Oa? 8 OL kee 
or, by Figure 11, 
ee See greene iar! 
Nese Me ge yc We. ts 
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If we use the general expression (2) and the fact that the 2kth row 
consists of 2k + 1 numbers, of which k are to the right and & to the 
left of the middle term, we obtain the formula 

1 2k 2%&—-1 %#k-2, k+l) 


will ole seen 3 
ic ame eae 2 3 k @) 








The value of wo, can be easily calculated using this formula, pro- 
vided k is not too large. If k is very large, it is extremely difficult to 
calculate this fraction. (Try, for example, to calculate wio,oo0!) We 
can estimate wz, by making use of the following remarkable 
inequality: 

l 1 


ag age te (4) 


To prove this inequality, we first transform formula (3): 
































1 2k 2k—-—1 2%wk-—2 k+1 
Wok = zea’ . eee oe 
aed 2 3 k 
tte Me Beh eS i ee ee 1 
~ 22k | 2 3 kK k k—-1 1 
_— 1 1-3+5-----Qk — 1)-2+4-6+--- 2k 
~ 92k | ras Sears arms (aes eee ee k 
aa i eee Og ee 
— 2k 1 2 3 k P23 k 
PN pe Sp ec en ee” a ae 
2 23 kK 2 4 6 2k 
We now write the three products 
Be ie eee eae Mee ee 5) 
223 45 67 8 9 10 2k — | 2k” 
US SSP Oo. eh ee: 
22446 6 8 8 10 10 2k ] 
1,2,3,4,5,6,7,8,9 10. 2-1, % 
23 45 67 8 9 10 11 2k =o bk + 1 


under one another. It is easily seen that of the three numbers in any 
column, the second is at least equal to the first, and the third is at 
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least equal to the second. Hence, (5) has the smallest and (7) the 
greatest value. The middle product is equal to 


GY GY GY Bae) =m 


the upper, after simplification, is equal to =p and the lower is 








equal to . Hence, 


! 
2k + 1 
Then, certainly, 


or, if we take the square root, 


1 ] 
pS WeE 
Vak ~~ 3k 
which was to be proved. 


By the same method, still more exact approximations for wa; can 
be found. 


Problem 10. Prove that, for all k > 2, we have 


Gitom< AOE o 


for all k > 3, we have 





Hint. In each of the products (5) and (7), change the initial 
terms in such a way that they coincide with the initial terms of the 
product (6). 


Problem 10 yields a sequence of increasingly accurate approxi- 
mations for w2,; the ratio of the lower to the upper bound serves 
as a measure of the accuracy. For approximation (4), this is equal 


to Jt for approximation (8), it is equal to Jt for approxi- 


mation (9), it is equal to Jz and for approximation (10), it is 





equal to 1 —- Fa” and, thus, approaches | as a limit. If the 
a 
products in the inequalities (8) and (9) are calculated out, we obtain 
] 
< Wor < ——=—._ (fork > 2), 11 
(sc een eo 
] 








Jaaok < Wor < Va eae (for k > 3). (12) 


On substituting the values 4, 5, 60, 150 for a in the general in- 
equality (10), we obtain 























aa < Wor Laz (for k > 4), (13) 
<= < Wer < aE (for k > 5), (14) 
Tit < Wo < ane (for k > 60), (15) 
Tage Sv < ae (for k > 150). (16) 


The coefficients of k constantly decrease on the left side of the 
inequality, while they constantly increase on the right side. One 
can prove rigorously (we shall not do so here) that both of these 
sequences converge to the same limit, and that this limit is the 
well-known a (the ratio of the circumference of a circle to its 
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diameter). Hence, w2, can be calculated for large values of 2k by 
the following approximation formula: 


(17) 


vats tl 
Wo ~ —=. 
Vak 

One can prove (and the reader can verify this for himself) that 
for k = 25 this formula yields an approximation that is correct to 
two significant figures; the larger the value of k, the more accurate 
the approximation. 


Problem 11. Calculate wio 000 to two significant figures. 


In the triangle of probabilities, the numbers increase as we move 
from the edges to the middle. Hence, by using the inequality (4) it is 
not difficult to find the upper bound of the numbers in the nth row. 


Problem 12. Prove that all elements of the nth row of the triangle 


of probabilities are less than or equal to = 
A 


7. ESTIMATION OF ARBITRARY ELEMENTS OF THE TRIANGLE 


Since we have found an approximation formula for the terms 
lying in the middle of the triangle of probabilities, it is only natural 
to seek an equally convenient formula for the other terms of the 
triangle. 

Consider the 2kth row. We denote the middle term of this row 
by vo (above, we called this term w2;,) and number all of the terms 
to its right in this row 


Vo, V1, V2, --+» Vk-2, Ve—-1, Ve. 
Now, the elements on the right side of the 2kth row of the quotient 
triangle are 


ts oa a eee 
Rael? eee? ee Oe 


By the definition of the quotient triangle, we have 


V1 k v2 k— 1 : . Vk-1 Z Ve l 





Go Ra Wy, ee es, Dee ea 
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If we choose a number s between 0 and k, and estimate the ratio 
Vs 


—, we obtain 

Vo 
Vs _ Vi Ve. Vs-1. Vs 
Vo Yo V1 Vs-2  -Vs-1 


_— kk k-l,  kK-s+2 k-stl 
ae on ee ae ee) an a ee | kts — 
If we reverse the order of the denominators appearing in these 
factors, we get 
a k k-—1 k= S82 k= tel 
Ve OK ES Ke se eH I ee? k+1 


= (ets) -rgiaa) (-r) -e) 


It is easy to see that the first factor is greater than each of those 
following it, while the last factor is less than each of those preceding 
it. It follows that 

















S 7 v S S 
— pl : 18 
(! a | a) ” 
or, 
Sey Vs ( k } 
es ee . 19 
( k+1 sy = k+s oe 


Problem 13. Between what limits does the probability Z129?° lie? 


One can find analogous estimates for the elements in the odd- 
numbered rows; however, we shall not concern ourselves with this. 


8. THE LAW OF THE SQUARE ROOT OF n 


Assume that our experiment of tossing the coin and moving the 
marker is continued for sufficiently long, say for 1,000 moves. How 
far is the marker then from the starting point? In any case, not far- 
ther than 1,000 steps to the right or left. And this is the only thing 
that we can state with absolute certainty. If we seek to assert that 
the marker moves less than 1,000 steps, that is, not more than 998 
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Steps, then an error is not impossible, for the same side of the coin 
can come up every time in 1,000 tosses (heads or tails), and the 
marker will then be found at — 1,000 or at 41,000. The probability 





sso l 
that this happens is, however, so small (it is equal to 2+) 


that we can regard such an outcome as practically impossible. 
Likewise, the probability that the marker passes the nine hundred 
ninetieth mark is small. For this to happen, the marker must reach 
one of the points 


— 1,000, —998, —996, —994, — 992, 

+992, +994, +996, +998, +1,000. 
To calculate the probability of this, we must consider the thousandth 
row of the triangle of probabilities and form the sum of the five 
outer left and the five outer right terms. Because of the symmetry 


of the triangle, this sum is equal to twice the sum of the five outer 
left or right terms: 


2/ 1 1 1,000 l 1,000 999 








71,000 * 71,000" ] 71,000 °°] 7 


1 1,000 999 998 
T1000 4 oe 3 
11,000 999 298 | 997) s 


TTR EC THI Waar 


290 zeros 


+ 





Hence, we can regard an error here as practically impossible. This 
is not surprising, for when it is assumed that after one thousand 
moves the marker has not passed the 990th mark, of the 1,001 pos- 
sible positions we neglect the ten most improbable ones. The state- 
ment that the marker remains under the hundredth mark (that is, 
that it does not pass the ninety-eighth mark) is much more risky. 
Here, we neglect the greater part of the possible positions, namely, 
902 out of 1,001. How justified are we in doing this? How small 
is the probability of an error in this statement or, on the other 
hand, how close to one is it? This question cannot be answered 
without calculation. To carry it through, we must find the sum of 
the 451 left outer and the 451 right outer terms in the thousandth 
row of the triangle of probabilities. While no knowledge is neces- 
sary for this calculation except that of the four fundamental 
arithmetic operations, few of our readers could carry it through to 


27 


the end. It would take too much time and energy.! We shall there- 
fore seek to estimate the probability, and forego an exact calcula- 
tion. Our considerations will be altogether general. To simplify the 
calculations, however, we shallsassume from now on that the num- 
ber n of steps that the marker makes is even. 

Let us consider the 2kth row of the triangle of probabilities and 
number its terms from the middle term to the right edge 


Vo, V1, V2, ..-5 VR. 
To estimate the sum 
S = Vp + Vega + +++ + Vk, 


we use inequality (19) of section 7. 
By this inequality, we have 


v k \ 
eed 
Yo k+r 


meth < ( k y" 
Vo Akt rel : 











; k 
For brevity, denot 
or brevity, we denote — 3 


fraction enclosed in parentheses on the right-hand sides of our 
inequalities exceeds g. Hence, 


by g. One sees immediately that no 


Vy Vr44 Vr42 Vr 
—< 2", tt < grt, — < grt2, Reece — < gk. 
Vo Vo Vo Vo 


1 Naturally, we can calculate this sum in the following way: We subtract from one the 
sum of the 99 middle terms. The number of summands in this sum is significantly 
smaller (99 instead of 902), but these summands themselves are considerably more 
difficult to calculate. 
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Adding these inequalities, we obtain 
S ’ 
Ge Pe ee ee 
0 


The right-hand side forms a geometric series, whose sum is 


— gkt1 
ee thus, 
Ss c Fs _ gett 
% |l-g 
But, g = ce <1, so that 1—g>0, and the inequality is 


k+1 


strengthened when we omit the negative number —5 from 





the right-hand side. Hence, 


If we multiply this inequality by vo and substitute the original 
value for g, we finally obtain 


S<wAt" (ey 20) 








Now we are in a position to estimate the probability P that 
after n = 2k moves, the marker is not less than m = 2r steps from 
the starting point. This probability is equal to twice the probability 
that after 2 = 2k moves, the marker is not less than m = 2r steps to 
the right of the starting point. The probability of this last event is 
precisely the sum S, which we have already estimated. In formula 
(20) we replace vo (for the middle term of the 2kth row) by the more 
convenient symbol wex, used previously; this reminds us that the 
middle term depends on the number of the row. We multiply both 


sides of inequality (20) by 2, replace k by 5 and r by 5° and thus 
obtain 


™m 


2 
P <2, AEM (1) (21) 
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The middle term 
Wn = Wak 


can be estimated by either formula (4) or (11)- a of section 6. 
By these formulas, we have 


1 
Wor < ——=, (22) 
VBR 
where B is a number which can be chosen as near to 7 as desired, 
provided k is large enough. (In (4) B = 2; in (11) B = 2.66, etc.) 


We replace k by 5 in the inequality (22). Comparing this in- 


equality with (21), we obtain 


m 


2 n+m ( n 3 
P< 2 —— - (—_—__} , 23 

Bn om n+m 
and can use this formula to calculate the numerical example dis- 
cussed at the beginning of this section. Suppose the marker makes 
1,000 moves. What is the probability that it is not less than 100 
steps from the starting point? We substitute n = 1,000 and m = 100 


in formula (23): 
2 10 \5° 
== 1 000n = ae 


Since k = ea 150, we can set B = 3.14 according to formula (16) 
of section 6. We then obtain 
P < 0.0048. 


The event that the marker is at least 100 steps from the starting 
point after 1,000 moves is thus highly improbable. If a not too high 
degree of certainty is demanded and, say, 0.005 is permitted as the 
degree of uncertainty, one can say that it is practically impossible 
that the marker has moved more than 98 steps after 1,000 moves; 
that is, it is practically certain that the marker has moved less than 
100 steps. 

For the further study of the motion of the marker, it is conven- 
ient to replace the approximation (23) by another that is not so 
precise but is simpler and more convenient to calculate. We require 
a preliminary inequality, the proof of which is left to the reader. 
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Problem 14. Prove that for arbitrary positive p and integral positive r, 


Al + py > 1+ 2p. (24) 
Let m and n be positive even numbers. We substitute r = = 


and p = in the inequality (24) and obtain 





a 2 2 
I m\ >], 4 ym 
( 5 a 2n we 2n 
Hence, 
n a 1 2n 
Gam ee ae Q5) 


From inequalities (21) and (25) it follows that 
2n 
m 


According to inequality (4) in section 6, w, < a furthermore, 
n 


m <n. Hence, the estimate 





pay 


follows from inequality (26). 
Thus, if it is asserted that the marker is less than m steps distant 
from the starting point after n moves, the probability of error is less 


than ( : vey 
m 

We choose an arbitrary positive number ¢ and estimate the 
probability of an error in the following statement: 

(A) After n moves the marker is less than ¢\/n steps away from 
the starting point. 

We denote by m the smallest even number that satisfies the 
condition 





m>ty/n. 
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Since the distance of the marker from the starting point after an 
even number of moves is an even number, the statement (A) is 
equivalent to the following statement: 

(B) After n moves the marker is less than m Se away from 
the starting point. 

Consequently, the probability of an error in the statement (A) 
is equal to the probability of an error in the statement (B). This 
probability is smaller than 


Cry <a) = @) 


Hence, we have proved the following important law: 








Law (The Law of the Square Root of n). With probability of 


3 
error less than (2) , one can assert that after n moves, the distance of 


the marker from the starting point is less than ty/n (that is, the 
marker is situated between —t\/n and +tv/n). 


We choose a certain degree of uncertainty, for example 0.005, 
and determine / so that 


(2) = 0.005. 

t 

As a solution of this equation for 1, we find? 
pS 219, 


¥/0.005 


It follows from the law of the square root of n that, for every value 
n, the statement that the marker has moved less than 12 \/n steps from 
the starting point in n moves is practically certain. 

We compile the following table: 


2,500 
10,000 
250,000 
1,000,000 





1 Here the approximation is greater than the actual value. 
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The second column of the table gives almost certain bounds on 
the distance of the marker from the starting point for various values 
st: 12 ; 
of n. The ratio — = —= approaches zero as n increases without 
no fn 
bound. 


Let us now assume that the marker is m steps distant from the 
Starting point at the end of nm moves. We call the ratio " the 


reduced velocity of the marker. If a particle starts out from the 
point 0 with this velocity and does not change direction, then at 
time n, its displacement amounts to m steps. 

For example: If the marker is at the point —20 after 100 steps, 
its reduced velocity amounts to 4. The reduced velocity varies be- 
tween 1 (when the marker always moves in one direction) and 
0 (when it returns to the starting point, that is, when it makes 
exactly as many moves to the left as to the right). From the /aw of 
the square root of n one easily deduces: 


THEOREM. When the motion of the marker is continued sufficiently 
long, it is practically certain that the reduced velocity is close to zero. 


In fact, it is practically certain that the displacement of the 
marker is less than 12 \/n, and, hence, that its reduced velocity is 
less than a ae. If n is sufficiently large, this bound will be 

no Vn 
arbitrarily close to zero. 

Until now, we have taken 0.005 as the permissible probability 
of error. However, we can repeat our considerations without signifi- 
cant alteration for an arbitrary value e of this tolerance. As a result, 
we come to the following conclusion: 

One can say with probability of error less than e that: 

(a) The displacement of the marker after n moves is less than 
ary 

Se > 


(b) The absolute value of its reduced velocity after n moves is 


2 
less than ——— n. 
We / ie 


Problem 15. Let the permissible probability of error be 0.05. Give 
a practically certain bound for the displacement and for the reduced 
velocity of the marker after 1,000 moves. 
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Problem 16. Let a be an arbitrary positive number. Prove that it 
can be stated with the probability of error less than 





er 
ay/n 
that the reduced velocity of the marker after n moves is less than a. 


Problem 17. Determine the number of moves that are sufficient for 
the reduced velocity of the marker to be smaller than 0.01, with the 
probability of error not greater than 0.001. 


9. THE LAW OF LARGE NUMBERS 


We now recall that the marker moves in accordance with the 
outcome of the toss of a coin. If on n tosses of a coin there are 
/ tails and n — / heads, the marker moves / steps to the right and 
n — | steps to the left, finally reaching the point 


Pesos 


The reduced velocity of the marker after moves is given by the 
absolute value of 


21 —n l 
The fraction . characterizes the relative frequency with which 


tails comes up. 

Let a permissible probability of error be given. We know that 
for large values of nv it can be asserted with practical certainty that 
the reduced velocity is close to zero. It is clear from equality (27) 


that for a small reduced velocity 2< is approximately equal to 1, 


and the relative frequency is consequently near 4. In other words: 


If a coin is tossed very often, it is practically certain that the 
frequency with which heads comes up is approximately equal to 4. 

Roughly speaking, it is practically certain that heads comes up 
exactly as often as tails. A more exact formulation would be: 
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Choose an arbitrarily permissible probability of error ¢ and an 
arbitrarily small number a. If the number of tosses of the coin exceeds 


l 
eee. me 
it can be asserted with a probability of error less than e that the fre- 


quency with which tails comes up differs from 4 by less than a. 
The proof for this exact formulation is easily obtained from 


l 
part (b) on page 33. For n > eye we have 


Seve, 
and 


2 
Ve 
Va 
Hence, the absolute value of the reduced velocity of the marker is 


less than 2a, with the probability of error less than e. In this case, the 


gio-niy! 4 
n n 


Hence, it can be stated with the probability of error less than e that 








< 2a. 


reduced velocity is equal to the absolute value of 





25 does not differ from 1 by more than 2a, or in other words, that 


4 differs from 4 by less than a. 


Problem 18. How often must a coin be tossed so that it can be 
asserted with the probability of error less than 0.01 that the fre- 
quency with which tails comes up lies between 0.4 and 0.6? 


Suppose now that a die is tossed instead of a coin. How often 
does the six come up? If the same calculations and arguments are 
carried through for this new case as for the example of the coin, we 
obtain the following result: for a great number of tosses, the fre- 
quency with which a six comes up lies near 4 with practical certainty. 

We consider yet another experiment. An urn contains a balls, of 
which b are white and the rest black. A ball is drawn 7 times from 
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this urn, and is returned to the urn each time. How often will a white 

ball be drawn? One can prove that it is practically certain that for 

sufficiently many trials, the frequency with which a white ball is 
> 


. 


: b 
drawn lies near ae 


We now formulate a general result including all of the above 
formulations as special cases. 

Suppose that an experiment is carried out in which an event A can 
either occur or not occur (a toss of tails, the roll of a six, drawing 
a white ball out of the urn, etc.), and let the probability of the occur- 
rence of the event A be p. (In our examples p was equal to 4, , and 


> respectively.) Suppose this experiment is repeated many times, the 


result of each trial not influencing the results of the succeeding ones. 
Then, for a large number of trials, it is practically certain that the 
frequency of the event A will be approximately equal to the probability 
Pp of this event. 

This general formulation can be made more precise, exactly as 
in the formulation for the case of the coin. 

This result is essentially nothing but a restatement of a well- 
known theorem of Bernoulli! which sets forth the simplest form of 
one of the fundamental laws of probability theory, the law of large 
numbers. Here we cannot go into the generalizations of Bernoulli’s 
theorem. We only remark that the most important is due to the 
Russian mathematician P. L. Chebyshev. 

The reader will appreciate the great significance of the law of 
large numbers. By the statement that the frequency of occurrence 
of an event A approaches the probability of A with practical cer- 
tainty for a large series of trials, the law of large numbers makes 
possible the experimental determination of this probability. In many 
cases, the experimental method for the determination of a prob- 
ability is the only possible one. Furthermore, the knowledge of the 
connection between probability and frequency enables one to draw 
practical conclusions about the frequency of appearance of an 
event in a long series of experiments from the theoretically calcu- 
lated probability of this event. The connection between probability 
and frequency is fundamental in many applications of probability 
theory to physics, technology, etc. 


1 Jacob Bernoulli (1654-1705), famous Swiss mathematician. 
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3. Random Walks with Finitely Many States 


In the preceding chapter, we considered the simplest example 
of a random walk, a random walk on a line. The problems posed 
there were of the following sort: At a given moment, where could 
the marker be, what is the probability that it is at a given point, 
and how far from the starting point is it? In this chapter, we con- 
sider more complicated schemes for random walks, including, for 
example, the random stroll through the city and the children’s 
game mentioned in the Introduction. The problems related to these 
schemes differ somewhat from those investigated in Chapter 2. If an 
arbitrary point is chosen, we ask whether a particle can ever reachit, 
and if so, when. The first question can be answered with the aid of 
a general theorem (p. 46). An exact answer for the second, dealing 
with the question of the number of moves necessary to reach a 
given point with a given probability, can be found only in the sim- 
plest cases (see Problem 20 to follow). For the general case, we 
can give only an approximation for the necessary number of moves. 


10. RANDOM WALKS ON A FINITE LINE 


Let us make a slight change in the scheme for a random walk 
on a Straight line. We place reflecting barriers in the path of the 
moving particle at the points and mp (see Fig. 12). These barriers 


Cone ore 


my me 


Fig. 12 


cause particles that reach m; to move to my + 1 on the next move, 
and those that reach mz to move to m2 — 1 on the next move. The 
motion of the particle will thus take place between the points 
my, and mg. 
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We could also restrict the motion of the particle by placing 
absorbing rather than reflecting barriers at the points m; and mz. 
In this case, the particle upon reaching either the point m, or the 
point mz would remain there permanently. (We have already con- 
sidered this diagram in the solution of Problem 6.) 

Finally, we could place a reflecting barrier at one of the two 
points and an absorbing barrier at the other. 


Problem 19. Show that the probability that after n moves the par- 
ticle has reached the point m at least once does not depend on 
whether there is a barrier at m, and if there is, whether it is a 
reflecting or an absorbing barrier. 


We place a reflecting barrier at point 0. A particle makes n 
moves starting from the point 1. What is the probability that it 
touches the point 3 at least once? To 
calculate this probability we can, ac- i 5 rn 
cording to Problem 19, place an absorb- 7X, 
ing barrier at point 3 (Fig. 13). But then, a erat 3 
the event that after n moves the particle 
touches the point 3 at least once is the Fig. 13 
same as the event that the particle is at 
point 3 after n moves. (For, when it reaches this point it remains 
there.) We wish to calculate the probability d, that the particle is at 
the point 3 after n moves. By an, bn, Cn, we denote the probability 
of the events that the particle is at point 0, point 1, point 2, 
respectively. 

To be at point 0 after n moves (the probability for this is ap), 
the particle must be at point | after nm — 1 moves (the probability 
for this is b,_1), and then go from | to 0 (this occurs with proba- 
bility 4). By Property 6, we have 

] 


an = bn_1 . 2° (1) 


Similarly, with the aid of Properties 6 and 7, we obtain the relations 


bn = An-1° l + Cn_1+ 4, (2) 
Cn = ba-1 7 5, (3) 
dn = dn_1+ 1 + Cn_1° 4. (4) 
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Problem 20. Using relations (1)-(4), show that 


Pe Pee (3). (5) 
Equality (5) can be interpreted as follows: the statement that 
after 2k moves the particle has touched the point 3 at least once, is 
false with probability (3)*. How many moves must be made for this 
probability to be less than 0.01? 
We obtain ko = 16.006 as a solution of the equation (3)*> = 0.01. 
Thus, we conclude that, for 


k > 16.006, (6) 
the inequality 
3\k 
(3) < 0.01 (7) 


is satisfied. Hence, for every k > 17, that is, for every number of 
moves greater than or equal to 2-17 = 34, the particle reaches the 
point 3 with a probability -greater than 0.99. (Compare with the 
calculations on page 45.) 


11. RANDOM WALKS THROUGH A CITY 


We return to a consideration of the random walk through a city, 
which we have described in the Introduction (Fig. 1). We ‘ask 
whether and when the friends reach the intersection E. We shall 
prove: If the time of their stroll is without limit, they reach the 
intersection £, exactly as with all other intersections, with proba- 
bility 1. Furthermore, we shall estimate the probability that EF is 
reached in a given number of moves. 

We state Our argument in general form, not assuming that the 
city must necessarily be of the form shown in Figure 1. Suppose 
that a traveler goes through the city. If he reaches an intersection 
from which k streets go out, the probability that he chooses to con- 
tinue along the first street is pi, that he chooses the second, po, ..., 
and that he chooses the kth street, p, (the case that he chooses 
with a certain probability the street by which he arrived is included 
in this enumeration). We assume that the numbers pj, po, .-., Dr 
are different from zero, and that for a given intersection the prob- 
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abilities remain constant. This means that the traveler always 
chooses his further path from a certain intersection with the same 
probabilities, independent of the direction from which he entered 
the intersection and the numberof times he has traversed it. (In the 
example given on page I, we have k = 4 and p; = po = p3 = pa = 3 
for every intersection.) If the traveler reaches the edge of the oe. 
he turns around and comes back. We shall assume that he continues 
in this way indefinitely. We claim that, no matter how the traveler 
wanders, for each intersection, the probability that he comes to 
this intersection is 1 (regardless of the location of the intersection). 


Proof. Part |. Let Eo be any intersection; we shall show that 
the probability that the traveler comes to £p is 1. Let the remaining 
intersections be denoted by £, E2,..., Ep. 

Suppose the positive integer N and the number a > 0 are such 
that the probability of his arriving at Eo at least once after com- 
pleting N moves,! regardless of his path, is greater than or equal toa. 

We split the totality of moves the traveler makes into groups of 
N moves each. Let the first subsequence consist of the first to the 
Nth move, the second, the (N + 1)st to the 2Nth, etc. We denote 
by D, the event that after making all the moves contained in the 
first k groups, the traveler has come to Eo at least once. We denote 
by D; the contrary event (that the traveler has not reached the 
point £o at any time while making the moves contained in the first 
k groups). 

The aim of the first part of the proof is to establish the inequality 


P(Dx) < P(Dx-1) =): 


To do this, we consider the event F,_1): “After making all moves 
in the first k — 1 groups, the traveler = not reached Ep and is at 
E, after (k — 1)N moves.” 

The events Dy_1, Fy1%, Fyi®, ..., Fx_1© are pairwise mutu- 
ally exclusive; they also form a complete system. By Property 7, 


P(Dx) = P(Dx_-1) P(Dx|De_1) + PCF e-1) P(Da|Fa™) + ++ 
+ PUFp-1) P(Di|Fxa™). (8) 
Obviously, 
P(D;|Dx_1) = |. (9) 


1For brevity, we say that the traveler has made a “move” when he goes from one 
intersection to a neighboring one. 
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Furthermore, P(D;|F,-1) is the probability that the traveler 
reaches the point Eo with a move in the kth group, upon the con- 
dition that he was at £, at the beginning of this group. In other 
words: P(D;|Fx-1™) is the probability that Eo is reached at least 
once in N moves, if the path begins at E,. By the choice of a, this 
probability is greater than or equal to a, that is, 


P(D;|Fx_-1™) > a. (10) 
Likewise, 
P(Dy|Fe-1™) > a, «.., P(Ds|Fe-1) > a. (11) 
Hence, it follows from Formulas (8)-(11): 
P(De) > P(De-1) + P(Faa®)a + PUP, ®@)a + +>. + PUR -1)a 
= P(Dx-1) + [PCPe-1) + PU) + +> + PU a@)Ja. (12) 


But the sum of the probabilities of the pairwise mutually exclusive 
events Dy_a, Fri™, Fri, ..., Fy-1™ is equal to 1, since these 
events form a complete system (Formula (7) of section 3). Hence, 
the sum in the square brackets is equal to 


1 — P(Dz-1), 
and Formula (12) takes the form 
P(Dx) > P(Dr-1) + [1 — PWr-vle. (13) 

By Property 3, 

P(D;) = | — P(D,), 

P(D;_1) = 1 — P(Dx-1). 
From this and (13), we obtain 
P(D;) = 1 — P(Dx) < 1 — PWDx-1) — [1 — PWDe-i a 

= P(D,;_1) = P(D;_1)a — P(Dy_1) (1 = a). 

Proof. Part 2. We now seek numbers N and a possessing the 
desired properties. 

We choose an arbitrary intersection F;. The traveler can reach 
the intersection Eo from the intersection E; by moving along various 
paths. We choose one of the paths F,E;F, ... E,E,Eo. We denote 
the number of moves in this path by N;. Let us calculate the prob- 


ability a; that the traveler traverses the path E,E;E, ... E-EsEo if 
he starts at Ej. 


4] 


We denote the probability that the traveler goes from the inter- 
section E; along the path E,E; by pi. The probability pj; is the con- 
ditional probability that the traveler is at E; after the nth move, 
under the condition that he was at E; after the (m — I)st move. 
Then the probability that the traveler starting out from E; takes 
the path E;E;E;, (Fig. 14) is equal to pi; pj. Indeed, by Property 6, 


Ms Vx 
Uy 


Fig. 14 


this probability is the product of the probability that the traveler 1s 
at E; after one move and the probability (under this condition) 
that he is at E, after one further move. The first factor is equal to 
pi, the second is equal to pj; hence, the desired probability is 
equal to pi;pjx. In general, the probability that a traveler starting out 
from E; chooses the path E,E;E;, ... E-EsEo 18 pij Pix - . . PrsPso = M%i- 

We carry through this calculation for every intersection Fj, Fo, 
... E,. We thus obtain the numbers Ny, No,..., Ny; a4, de, ..., Ay. 
Let N be the greatest of the numbers Ni, No, ..., Ny, and a be the 
smallest of the numbers aj, a2, ..., ay. We must show that N and a 
have the desired properties. 

First of all, the numbers aj, ao, ..., ay are all positive; hence, 
we also have a > 0. 

Suppose the traveler has traveled N moves from the intersection 
E;. He has thereby reached the point Ep at least once, with a prob- 
ability greater than or equal to a; the event “the traveler took the 
path E,E;E;, ... E,E;Eo in the first N; moves” implies the event 
“the traveler reached Eo at least once in N moves” (for if the first 
event occurs, the traveler reaches Eo after having made N; moves). 
By Property 1, the probability of the second event is greater than 
or equal to the probability of the first, which is a; > a. 
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Proof. Part. 3. We estimate, successively, the probabilities P(D,), 
P(D2),..., P(Dx), .... ; 
First of all, P(D,) > a, and hence, by Property 3, 


P(D,) = 1 — P(D) < 1 ~<a. 
Furthermore, we find, with the help of the formula 
P(Dx) < P(Dx-1) (1 — 9), 
obtained in the first part of the proof, that 
P(D2) < P(D,)(1 — a) < (1 — a), 
P(D3) < P(D2)(1 — a) < (1 — a), 
pe nthe Bole he Beek Seat Gag Wd i ek (14) 


We consider the event D (that the traveler never reaches Eo). 
D implies each of the events Dy, Do, ..., Dy, .... Hence, P(D) is, 
by Property 1, not greater than the probabilities P(D,), P(D2), ..., 
P(D,), ... and, hence, not greater than any of the numbers 


Psat Say Salieecs (15) 


Since a > 0 and 1 — a < 1, the terms of the sequence (15) form an 
infinite decreasing geometric progression, which for increasing k 
becomes smaller than any arbitrary positive number e. Hence, the 
probability that Zo is never reached is smaller than any positive 
number, and is thus equal to zero. By Property 3, the probability 
that Eo will be reached is then equal to I. 

The following theorem has thus been proved: 


THEOREM. The traveler reaches every intersection with a proba- 
bility of \ regardless of the intersection from which he starts. 


Remark |. We consider two arbitrary intersections, denote them 
by Eo and £,, and assume that the traveler starts his path at the 
intersection FE. Let us calculate the probability of the events that 

Ao : The traveler reaches Eo at least once; 

Ao, : The traveler reaches Ep and then the intersection Fj; 

Aoio: The traveler reaches Eo, then reaches £,, and after that 

returns to Eg, etc. 
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By the theorem just proved, we have P(A) = |. By Property 6, 
the probability of the event Ao; is equal to the product of P(Ao) 
and the probability that the traveler, on going out from £o, then 
reaches £,. By our theorem, both factors are equal to 1; hence, 
P(Aoi1) = |. Furthermore, the probability P(4o10) is,’ by the same 
Property 6, equal to the product of P(Ao) and the probability that 
the traveler, on going out from £), finally reaches Eo. From this, it 
is obvious that 


P(Ao10) = 1. 
In a similar manner, we show that 


P(Ao101) = P(4o1010) = «+: = 1. 


We pick an arbitrary positive integer k. It follows from what has 
been proved that: 

a) With the probability 1, the traveler returns to the starting 
point at least & times. 

b) With the probability 1, the traveler reaches an arbitrary 
preassigned intersection Epo at least k times. 


Remark 2. The formula 
P(Dx) < (1 — a)é 


(compare with (14)) gives an estimate for the probability of error 
of the statement that after kN moves the traveler has reached Ep 
at least once. We choose an arbitrary probability «. Then, we can 
find a number & such that 


(Q-—ajF<e. 
But then, certainly, 
P(Di) <, 


and the statement that after KN moves the traveler has reached Ep 
at least once, has a probability of error less than e. Thus, even for 
arbitrarily great demands on the degree of certainty, one can give 
a number of moves in which the traveler reaches Eo with practical 
certainty. 


Formula (14) is general; that is, it permits an approximation of 
the probability P(D;) for an arbitrary real city. However, it yields 
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only a rough approximation to this probability. We shall apply this 
approximation to the consideration of an earlier example (see 
Fig. 13). In this example, one can take N = 3 and a = 1. Hence, 
the probability that after k+ 3 moves the traveler never reaches the 
point 3 satisfies the inequality 


P(Dx) < (3) 


We ask that 
3\k 
(=) < 0.01. 


The smallest value of k that satisfies this inequality is equal to 17 
(see p. 39). Therefore, the smallest suitable number of subsequences 
is equal to 17, and we reach the point 3 after 17-3 = 51 moves with 
a probability of at least 0.99. However, as was shown earlier, even 
34 moves are sufficient (not less). Thus, even in this simple case, our 
approximation yields a result which is less accurate than the exact 
calculation by a factor of one and one half. In more complicated 
examples, the result becomes still less precise. 

Our traveler need not go on foot; he can use one of the munici- 
pal methods of conveyance: streetcar, bus, trolley bus, or subway. 
At a stop there is a certain probability that he enters, say, a bus. 
After that, he gets off with a certain probability at each of the fol- 
lowing stops and continues with a certain probability. 

In this case, the random path of the traveler through the city 
does not differ in essence from the board game Circus mentioned 
in the Introduction. One can instead name the game Journey 
Through a City. The squares of the board are represented as street 
intersections, and municipal conveyances substituted for the circus 
attractions. The sudden motion to another edge of the board cor- 
responds, perhaps, to the journey to the next subway station. We 
wish to use a device of chance that offers more possibilities than a 
die. One can, for example, use an urn, and determine the direction 
of motion by drawing from the um a piece of paper bearing the 
designation of an intersection. It is clear, however, that a single 
urn will not suffice, since the point that we reach depends on the 
point from which we start. Ideally, one would use as many differ- 
ent urns as there are squares on the board. 
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12. MARKOV CHAINS 


We now consider an arbitrary diagram of points Fj, Eo,..., En, 
several of which are joined by arrows that point in the possible 
directions of motion. ¢ 


DEFINITION. A system of n points or states for which'we know the 
possibilities of transitions between them as well as the probabilities that 
these transitions take place, is called a Markov chain. 


The probability that one goes in one step from E; to £; is gen- 
erally denoted by pi; (in particular, pj; denotes the probability of 
not moving from £; on a move).! The transition from £; to £; is 
possible if pi; > 0 (in this case, we draw an arrow from E£; to £&)). 


DEFINITION. A Markov chain is called irreducible if one can go 
from any position E; to any other position E; by means of a chain of 
possible transitions. 


In terms of the arrows, this means , 


: ae eee ee 
that one can go from any point £; to 
any other point £; in the direction of 
the arrows. Figure 15 gives an example Pers 
E2 


-—<—O 
of a chain that is not irreducible (it is Ey 
impossible to go from Ez to EF, in the Fig. 15 
direction of the arrows). Figures 16 to 
20 are examples of irreducible chains. We have already considered 
a Markov chain of two states in Problem 5 (Fig. 4). 
The following general theorem holds: 


THEOREM. In the motion of a particle through an arbitrary system 
of states that form an irreducible Markov chain, this particle reaches 
any state with a probability of | (independent of its starting point). 


This theorem was already proved, in essence, as a theorem on 
the random walk of a traveler through a city. The reader can ascer- 
tain without difficulty that our proof is also valid, step for step, 
without an alteration for arbitrary irreducible Markov chains. 

The Markov chains are named after the noted Russian mathe- 
matician Andrei Andreievich Markov (1856-1922), who discovered 
1 We remark that, for every i, 

Put pi2t-+>+pat-:> + pan =1, 


SINCE Pi1, Pi2, ..-, Pin are the probabilities of mutually exclusive events (going from 
E, to E, from £; to £2, and finally from £; to E,) that form a complete system. 
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and investigated them. Markov chains are very important because 
of their applications to science and technology, as well as for prob- 
ability theory itself. : 

In the solution of Problem 5 it was remarked that for a chain 
of two states the probability that after n moves a given position is 
reached depends less and less on the starting point with increasing 
n. This fact holds for all irreducible Markov chains (with minor 
limitations). 


13. THE MEETING PROBLEM 


We return once more to the city plan in Figure 1. Suppose that, 
as before, the friends start out on their walk from intersection A, 
but that this time they go their own ways; that is, each of them 
tosses two coins independently of the other and chooses his path 
according to the outcome of his own toss. Will the friends meet 
again after they have left 4? We shall show that with probability 1 
this meeting will take place somewhere. Moreover, if any crossing 
E is fixed beforehand, it can even be asserted that with probability | 
the friends will meet at E. 

We restate this problem in general terms as follows: 


THEOREM. Two particles move along an irreducible chain K, be- 
ginning their motion at the same time at an arbitrary state. At each 
move, each particle moves independently of the other, from one state 
to another. The probability that the two particles meet in an arbitrary 
preassigned state is equal to 1. 


First, we explain an important relation 
between Markov chains. 


|] oL——— 02 


Let the chain K consist of n states Ey, Es, Fig. 16 
..., En. We consider n? points, and denote 
them by Fa, Exo, dbs Fin; Eo1, E22, ansis Eon, 022 
_.., Ena, Eno, .., Enn. We shall denote the 7 ea 


state of the pair of particles on the chain K by 

a marker that is situated on one of the points y \ 
Ey1,..., Enn. Uf the first particle is in state Fi, 

and the second is in state E;, we place the 120 ~——> 021 
marker on the point Ej,. The passage of the Fig. 17 


first particle from E; to E; occurs with the 
probability p;;, and the passage of the other particle from E;, to E; 
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with the probability p,:. The simultaneous passage of the first par- 
ticle from E; to E;, and the second from £, to E; occurs with the 
probability pi; px: (by Formula (4), section 2), and, hence, the marker 
goes from Ey, to Ey, with the probability pijpa. We have thus 
obtained a new Markov chain, which we denote by K?. The chain 
K? obtained from the chain K shown in Figure 16 can be seen in 
Figure 17. 

According to the general rule, we can draw an arrow from Ex 
to Ej, when the probability of the passage from Ej, to Ey, is positive. 
From this it follows that there is an arrow from Ej to Ey if and only 
if the two probabilities pi; and px; are positive, that is, if and only 
if an arrow leads from £; to E; and an arrow leads from E;, to £}. 
The system of arrows in the chain K? is thus formed from the sys- 
tem of arrows in the chain K, the value of p,; being unimportant. 


Problem 21. (a) The system of arrows of a chain K is shown in 
Figure 18. Construct the system for the chain K?; 

(b) Do the same for the system shown in Figure 19; 

(c) Do the same for the system shown in Figure 20. 


AA fs 


nearer 1o~————-03 


Fig. 18 Fig. 19 Fig. 20 


We consider the totality L of states in K? which one can reach 
from £;; by moving along the arrows. 


Problem 22. Prove that: 
(a) The positions E22, E33, ..., Enn are contained in L. 


(b) L is an irreducible chain. 


The proof of the meeting theorem can now be given in a few 
words. 
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Proof. Let K be an arbitrary irreducible chain, and suppose 
that the two particles begin their motion at the same time in the 
state £y,,. We construct the chain K?. This may not be irreducible 
(as was shown, for example, in Problem 215). We form the chain 
L from K?. This chain is (as was shown in Problem 225) irreduc- 
ible, and we can apply the general theorem formulated on page 46. 
Hence, after leaving the state Fy, the marker will reach any state 
of L with probability 1, including £41, Eee, ..., Enn. But this 
means that our particles meet with probability 1 in any arbitrary 
preassigned state. 


It is proved in exactly the same way that 3 or 4 or, in general, n 
particles that begin their motion at the same time at a state of an 
irreducible chain, meet again with probability 1, and furthermore, 
with probability 1 they reach any arbitrary preassigned state at the 
same time. 
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4. Random Walks with Infinitely Many States 


The Markov chains that we considered in the previous chapter 
had finitely many states. We now turn to a consideration of chains 
with infinitely many states. (We have already met one such chain 
in the diagram of the random walk on a line.) Exactly as in the 
previous section, the question of interest to us is whether a particle 
reaches a given point, and if it does, how fast. The random walk of 
a particle on an infinite chain differs qualitatively from the motion 
on a finite chain. For example, one cannot in general assert here that 
the particle reaches every position with the probability 1 (although 
this is possible in individual cases). 

We limit ourselves to the simplest chains with infinitely many 
states. First, we investigate the scheme, already familar to us, of the 
random walk on a straight line (we shall also call this chain an 
“infinite path”). Then, we shall consider another simple example of 
a chain with infinitely many states, an infinitely large city with 
a checkerboard pattern (Fig. 21). If a traveler reaches any crossing, 
he continues in any of the four directions with a probability 4. 


Fig. 21 
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14. RANDOM WALKS ON AN INFINITE PATH 


Problem 23. Let the points 0, +1, +2, +3, ... be marked on a line 
(Fig. 22). A particle located at a point n might move at the next 


moment to the point n + | with the probability 4 and to the point 
n — | with the same probability. At the beginning, the particle is 
situated at the point 0. Find: 

(a) The probability x that the particle reaches the point | at 
least once. 

(b) The probability y that the particle reaches the point —1 at 
least once. 

(c) The probability z that the particle returns to the point 0 at 
some time (that is, that it is situated at the zero point at a time other 
than the beginning of the random walk). 


Problem 24. Prove that the particle of Problem 23 reaches every 
point with the probability 1. 


r r r r r r r 
eee r(Ne@)eG) 
eee 
P P P P P P 
Fig. 23 


Problem 25. Suppose that in the diagram in Figure 23 the particle 
moves from the point n to the point n + | in a unit of time with 
probability p, moves to the point n — 1 with the same probability 
p, and remains in the same place with the probability r (in Problem 
23, p = 4nd r = 0). Prove that the particle reaches every point 
with the probability 1 for p > 0, regardless of the starting point. 
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It is established for the chains in Problems 24 to 27 that the 
particle, wherever it might start, reaches every position with prob- 
ability 1. However, the chain shown in Figure 24 does not have 


» 
x 
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Gq 4 q 


Fig. 24 


this property; here, the particle goes from n to n + 1 with the 
probability p, and from n to n — | with the probability g = 1 — p, 
where p > gq. One can prove (we shall refrain from doing so here) 
that the probability that the particle ever reaches the point —1 
after starting out from the point 0 is less than 1. 


Problem 26. Using the fact that in Figure 24 the probability of ever 
reaching the point —1 from the point 0 is smaller than 1, find the 
probabilities x, y, z (of Problem 23). 


In Problem 23, it developed that a particle moving on the line 
returns to its starting point with a probability 1. How many moves 
must it make to make this return practically certain? The calcula- 
tion shows that, for a probability greater than 0.99, several thousand 
moves are necessary; if this probability is to be greater than 0.999, 
then several hundred thousand moves are necessary. 

One can show that the probability of a return is less than 1 — € 


when the particle has made less than Se moves. When it has made 


more than 





moves, this probability is greater than | — e. 


It was further shown that the particle reaches every point with 
probability 1. We ask again, how many moves are necessary for the 
particle to have reached a given point with practical certainty. It is 
clear that the farther the point of interest to us is from the starting 
point, the greater the number of moves necessary for the particle 
to reach it. To reach the point 10 with a probability of 0.99, more 
than 100,000 moves are necessary; to reach the point 100 with the 
same probability, tens of millions of moves are necessary. 
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We set 





Nyx 20k = De 
e2 
2 
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One can prove that the probability that the particle reaches the 
point 2k even once is smaller than | — e, if it makes less than M4 
moves. If it makes more than Nz moves, the probability of the same 
event is greater than | — e. 


15. THE MEETING PROBLEM 


We now turn to the problem of determining the probability that 
two travelers on an infinite path K will meet (Fig. 22). In the pre- 
vious chapter, we solved the analogous problem for an arbitrary 
Markov chain with finitely many states by reducing it to the prob- 
lem ofa single particle moving on a new, more complicated Markov 
chain with finitely many states. This device was sufficient for the 
solution of the meeting problem, since we had a general theorem 
about the random motion of particles that held for all arbitrarily 
complicated irreducible Markov chains with finitely many states. 
In the case of infinitely many states, the meeting problem cannot 
be solved in this fashion, since we lack a corresponding theorem 
holding for all chains with infinitely many states. We are therefore 
compelled to solve the meeting problem by a direct method. 

In the case of infinitely many states, we cannot reduce the 
motion of two particles to the motion of one particle on a more 
complicated chain. On the contrary, we must reduce the problem 
of motion of a particle on a complicated chain to a problem on the 
random motion of two (or several) particles on a simpler chain. We 
shall proceed in this way in the following investigation of an infi- 
nitely large city with a checkerboard pattern. We shall reduce the 
motion of a particle (marker) in this city to the motion of two 
particles (travelers) on an infinite path K. 

We make use of the following auxiliary problem. 


1 This statement holds fork > 2.€< 2 _ 


1 
= 
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Problem 27. Prove that the sum 


l ree l 1 
OS a erg he 
k Foes ta aa aa tae 


increases with k and can surpass‘any arbitrary value. 


THEOREM. Suppose the travelers begin their journey at the same 
time at point 0. Then they meet again at this point with probability 1. 
Proof. Part \. We consider the events: 
As: Both travelers reach the point 0 after s moves. 
B,: Both travelers reach the point 0 after s moves, without 
a previous meeting! at this point. 
C,: Up to and including the sth move, the travelers have not 
met at the point 0. 
The events B,, Bo, B3,..., Bs_i1, Bs, Cs are pairwise mutually 
exclusive and form a complete system. Hence (Property 7), 
P(4,) = P(B1) P(As|B1) + P(Bo)P(A,|Bo) + +> 
+ P(B;) P(A;|Bs) + P(Cs) P(As|C.). (1) 
Clearly, 
P(A,|C;) = 0, P(A;|Bs) = 1. 
Furthermore, P(A,|B;) is the probability that both travelers are at 
point 0 after the sth move, under the condition that they had already 
met there after the ith move; that is, P(A,|B,) is the probability that 
the two travelers, starting out from 0, return there after s —i 
moves. Hence, 


P(A,|B;) = P(As-_i). 
If we now set 
P(A;) = as, 
and 
P(Bs) = bs, 
equality (1) assumes the form 


as = byas_4 + beds—2 tect + bs_141 + De: 


1 Except, of course, at the initial moment. 
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We write this equality out for s = 1, 2, 3,..., m and add (we write 
out those a;, b, that are equal to zero for the sake of clarity): 


a= bi, ~ 
az = bo + aybi, 
a3 = b3 + aybe + ach, 


On = bn + Aybn-1 + Gebn_2 + +++ + Gn-sby 


Rn = Qn + 41Qn-1 + G2Qn-2 + +++ + 4n-101. 
Here we have the set 
Ry = a1 + G2 +43 +--+: + Gp, 
QO, = by + bo + bg +--+ + by. 


Clearly, Q, is the probability of the event D,: “in the course of 
the first n moves, the travelers meet at least once at the point 0”; 
for we have 


D, = By + Bo + By + --- + Br, 
P(Dn) = P(B1) + P(B2) + --- + P(B,) 
= bj) +be+-+- +b 
= 0%: 


The probability Q that the travelers meet at 0 at some time (the 
probability we are seeking) is, by Property 1, greater than or equal 
to Q,; that is, 


Q> QO (3) 
for every n. 
It follows from (2) and (3) that 
Rn <QO+ 40+ 4204+ --+ + 4n10 = (1 + Rn-1)Q, 
Rn 


———_.. 4 
l + Ra-1 ( 


Q2 
The inequality (4) holds for every n. 
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Proof. Part 2. We shall now show that R, exceeds every arbi- 
trary number with increasing n; we have R, = a; + 42+ 434+ -:: 
+ a», where a, is the probability that the two travelers meet at 0 after 
n moves. Since the two travelers move independently of each other, 
a, can be found by the multiplication formula for probabilities: 


an = Wr?, 


where w, is the probability that one traveler is at point 0 after 
n moves. The probability wo.41 is, obviously, equal to zero. As was 
shown in Chapter 2, 


Hence, 


1 
42x41 = 0, do, > Ak’ 


From this it follows that 


1 
Ronyi = Rox > qn 


where S; = 1 + 1 of: 1 Be ache: DE i Since, however, S; increases 


we. 3 k 
without bound, R,, also increases without bound. 

Proof. Part 3. We now prove that Q = 1. First of all, Q, like all 
probabilities, is either less than or equal to 1. We assume that 
Q = 1-—d, where d>0, and come to a contradiction. By in- 
equality (4), 


1d > — Re __ _ Rat + Gn 
~ 1+ Rn-1 Raa +1’ 


Rn-1 + 1 — dRa-1 + 1) > Ra-1 + G, 
1a SOR 21); 
] a ARn_1, 


; 1 
Rn-1 < a’ 
But this is false, since R,_1 increases without bound for increasing n. 
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16. THE INFINITELY LARGE CITY WITH A CHECKERBOARD PATTERN 


We have already said. that we shall reduce the motion through 
an infinitely large city with a checkerboard pattern to the motion 
of two travelers on an infinite path K. 

Let us imagine that two travelers move on the path K. The chain 
K? is constructed by the process given in the preceding chapter, and 
consists of the positions (0, 0), (0, 1), (0, —1), Gd, 1D, Gd, —)), 
(-—1, 1), (-1, —)D), etc. The chain K? is shown in Figure 25. 


»3)x 
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Fig. 25 


Suppose that the travelers are at the points m and n of the chain 
K. The first traveler goes with a probability $ from m to m + 1, and 
the second traveler goes with the same probability from” ton + 1. 
Hence, the probability that the two travelers go from the point pair 
(m, n) to the point pair (m + 1, + 1) is equal to4-4 = 4. This 
means that the marker that moves on the chain K? and reflects the 
position of both travelers on K goes from (m, n) to (m + l,n+ 1) 
with the probability 4. The marker goes from (m, 1) to each of the 
other points 


(m+1,n—1),(m—1,24+),(m—1,n— 1) 
neighboring (m, n) with the same probability 4. 
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We remark that at each move the distance between the travelers 
either changes by two or not at all. Therefore, if at the beginning 
the travelers are at the points k and /, they can subsequently reach 
at the same time only such points m and n whose difference m — n 
is of the same parity as k — /. Hentce, the chain K? is not irreduci- 
ble; it divides into two irreducible chains: into L, containing the 
pairs (m, n) with even difference (that is, the points that in Figure 25 
are joined by thick lines), and M, containing the pairs with odd 
difference (that is, the points in Figure 25 that are joined by thin 
lines). 

We assume that the marker starts out from the position (0, 0) of 
the chain L (corresponding to the fact that both travelers start out 
from the position 0 of the chain K at the same time). The marker 
must remain on the boundaries of L in its further motion. The chain 
L is shown separately in Figure 26, where it is rotated 45° from its 


N_2 N_y N Ny N2 





Fig. 26 


position in Figure 25. However, Figure 26 coincides with Figure 21. 
We find, therefore, that the problem of the random walk of two 
travelers on the path K is equivalent to the problem of the random 
walk of one traveler through an infinitely large city of checkerboard 
pattern. 

We have proved that two travelers starting out from the point 0 
of the path K at the same time, will, with probability 1, again meet 
at this point. In terms of the city L, this means that a marker that 
begins its motion at (0, 0) returns there with a probability 1. Thus, 
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simply translating the result of the previous section into this new 
language, we have proved that the marker returns to the starting 
point with probability 1. ~ 

The return of the marker to the starting point takes much longer 
in the infinitely large city with a checkerboard pattern than on the 
infinite path; that is, many more moves must be made for this 
return to be practically certain. 

For example, to obtain a probability of return of more than 0.99 
the marker must make an astronomical number of moves: more than 
1088 (in the case of the path K, 10,000 moves sufficed). 


9 

In general, if the marker makes less than Oe n17 moves, the 
probability that it returns at least once to the starting point is less 
than | — e. This probability becomes greater than 1 — e when the 


marker makes more than 102-16 moves. 

We now prove that the marker moving about the infinitely large 
city with a checkerboard pattern not only returns to the starting 
point with probability 1, but also reaches any arbitrary preassigned 
intersection with probability I. 

Let Eo and £, be two neighboring intersections. By x, we 
denote the probability that the figure reaches the point F, at any 
time, having started out from Ep. Due to the uniform construction 
of our city, this probability has the same value for any pair of 
neighboring intersections. 

Suppose that the marker starts out from Eo. After the first 
move it can have reached one of the four intersections 01, 02, 03, 04 
next to Eo. We consider the events: 

A;: After one move, the marker reaches the point Qj. 

B: The marker returns at any time to Epo. 

By Chapter |, Property 7, 


P(B) = P(A1) P(BIAx) + P(A2) PBA?) 
+ P(A3)P(B\As) + P(A4) P(BIAy). (5) 
From the plan of the city, one sees that 


P(A) = P(A2) = P(As) = P(Aa) = + 


The occurrence of B under the condition A; means that the marker 
reaches the point Ep on starting out from the point 0; (to which it 
came after the first move). 
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Hence, 
P(B\A,) = P(B\A2) = P(B\A3) = P(B[A4) = x. 


Finally, as was proved above, P(B) = 1. If all these values are sub- 
stituted in Formula (5), we obtain” 
l ] ] l 
LS get ae eg 
and thus, x = l. 

Now, let £ be an arbitrary intersection of the checkerboard city. 
We consider one of the paths that lead from Ep to E, and number 
in order all of the intersections that belong to this path: Eo, 
PA Boy. cg Bg aE, 

By D we mean the event that the marker, on starting out from 
Eo, reaches Ej, then £o, etc., and finally, FE, = E. 

The probability that the marker reaches E;;1 at some time after 
starting out from £; is, as was calculated, equal to 1. If we multiply 
these probabilities for i = 0, 1, 2,...,n — 1 according to the mul- 
tiplication formula for probabilities, we obtain P(D). Clearly, the 
occurrence of the event D implies the occurrence of the event C, 
that the marker reaches E at some time. 

By Chapter 1, Property 1, 


P(C) > PD) = 1, 


and, consequently, P(C) = 1. 

The consideration of the random walk through a city of the kind 
given makes it possible to sharpen still more the formulation deal- 
ing with the infinite path. It was shown that the travelers moving 
on the infinite path meet at the starting point with a probability 1. 
We can now assert that the travelers meet at any arbitrary preas- 
signed point n with the probability 1. For this is equivalent to the 
statement that a marker moving in the city with a checkerboard 
pattern reaches the intersection (, n) with probability 1. 

This result, however, cannot be generalized to arbitrarily many 
travelers as in the case of a finite Markov chain. For four or more 
travelers moving on the path K, the probability that they all meet 
is smaller than 1. For three travelers, the probability that they all 
meet somewhere is equal to 1, but the probability that they meet in 
a preassigned point is less than 1. 
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We can summarize the results of our investigations of the prob- 
lems on motion and meeting in a city of checkerboard design in 
the following theorems: * 


THEOREM 1. Let A and B be two arbitrary points on the path K. 
Two travelers that start out from the point A at the same time meet 
at the point B with probability 1. 


THEOREM 2. Let A and B be two arbitrary intersections in the city 
L. A traveler starting from the intersection A reaches the intersection B 
with probability 1. 


The proofs of these theorems are intimately connected. The 
proof of the second theorem arises from the special case of the 
first, in which the points A and B coincide. The general formula- 
tion of Theorem | arises, in turn, from Theorem 2. 

The special case of Theorem | that constitutes the first link in 
our chain of reasoning demands for its proof some calculations. 
These calculations can be avoided if one contents himself with a 
weaker assertion than that of Theorem I. 


THEOREM la. Two travelers starting out at the same time from an 
arbitrary point n of the path K meet again with probability |. 


If we carry over this statement to the context of our city, 
it assumes the following form: 


THEOREM 2a. A traveler starting out from the intersection (n, n) 
of the city L reaches, with probability 1, a point of the line NN 
(Fig. 26). 

The proof of this last theorem can be carried through with the 
aid of Problem 25 alone. Hence, we pose the following problem: 


Problem 28. Deduce Theorem 2a with the aid of Problem 25. 
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Concluding Remarks 


In this booklet we have considered problems in probability 
theory. It was our aim to acquaint the reader with the concepts 
and methods of this unique science by means of examples that were 
sufficiently clear and which, at the same time, required for their 
consideration more complicated processes than the simple calcula- 
tion of the number of desired outcomes. The fact is that the over- 
whelming majority of the problems that are dealt with by modern 
probability theory cannot be solved by such simple calculations. 

On the other hand, we could not consider more complicated and 
interesting examples, since they require for their solution tools that 
lie outside the realm of elementary mathematics. However, we do 
not wish to leave the reader with the impression that probability 
theory is the science of children’s games and strolls through a city. 
In reality, the domain of application of probability theory is very 
great. Probability theory is widely used in technology (radio engi- 
neering, the design of telephone networks, quality control of pro- 
duction, etc.), ballistics (the investigation of the scattering of shots), 
the evaluation of experimental results (the theory of errors). Prob- 
ability theory also finds important and diversified applications in 
physics. We have already spoken of some of them (Brownian motion). 

Beginning with the work of the great mathematician P. L. 
Chebyshev (1821-1894), Russian scientists have taken a leading role 
in the theory of probability. Chebyshev’s work has been continued 
by his students, A. A. Markov (1856-1922) and A. M. Lyapunov 
(1857-1918). The Soviet school of probability theory has produced 
such distinguished scholars as S. N. Bernstein (1880- ), ALN. 
Kolmogorov (1903- ), and A. Ya. Khinchin (1894-1959). 
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Applications. Vol. 1. 2d ed. (New York: John Wiley & Sons, Inc., 
1957.) 


Gnedenko, B. V., and Khinchin, A. Ya. Elementary Introduction 
to the Theory of Probability. Translated by W. R. Stahl. (San Fran- 
cisco and London: W. H. Freeman and Company, 1961.) 


Goldberg, Samuel. Probability, an Introduction. (Englewood 
Cliffs, N.J.: Prentice-Hall, Inc., 1960.) 


Wolf, Frank L. Elements of Probability and Statistics. (New 
York: McGraw-Hill Book Company, Inc., 1962.) 
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Solutions to Problems 


PROBLEM 1. There are 7 possible events, of which 3 satisfy the condition. 
Hence, the desired conditional probability is equal to 3. 


PROBLEM 2. The probability that a six does not come up on one toss is equal 
to 2. The probability that no six appears on six tosses is, by formula (6) on 
page 10, 


535 (55 Sha JS 2 15,025 535 
666666 ie) ~ 46,6567 


PROBLEM 3. We denote the desired probability by x. Then, the probability of 
no hits in # shots is equal to 1 — x, of no hits on one shot is equal to 1 — p 
(Property 3). Exactly as in the previous exercise we have, by formula (6), 


1-x=(1 —py, 


and thus, 
x=1-—(1 —p)*. 


PROBLEM 4. We denote the probability of the event A, that the first player 
wins, by x, the probability of the event B, that the second player wins, by y, 
and the probability of the event C, that the game never ends, by z. 

The events A, B, and C are pairwise mutually exclusive and form a com- 
plete system; hence, 


x+y+z2z=P(A) + P(B) + PCO = 1. (1) 
The probability that the second player wins is equal to the probability 
that tails comes up on the first toss (this probability is equal to 4) multiplied 
by the probability that this player wins under this condition (Property 6). 
Then, however, the second player is in the position of the first, and, hence, 
this conditional probability is equal to x. Thus, 
y = $x. (2) 
The probability that the first player wins is found by the formula for the 
complete probability. We consider the events: 
D: “Heads comes up on the first toss.” 
F: “Tails comes up on the first toss.” 
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The events D and F are mutually exclusive and form a complete system; by 
Property 7, we have 
x = P(A) = P(D)P(A|D) + P(F) P(A|F). 


Obviously, P(D) = P(F) = 4 and P(A|D) = 1. To calculate P(A|F), we 
remark that the first player is in the position of the second, and the second 
player in the position of the first, when tails comes up on the first toss; the 
probability of his winning is then y. Hence, 


eth wy AL 
£25 PS) (3) 


If we solve the equations (2) and (3), we find x = 4 and y = 4. We obtain 
z = 0 from equation (1). 

The fact that the probability of the game continuing indefinitely without 
an outcome is equal to zero can be obtained by direct calculation (exactly as 
can be done for the probability of the event that no six comes up in rolling 
a die). 


PROBLEM 5. (a) We denote by pii™ the probability that the particle, on 
starting out from A, is again at A after n moves, and by pi2™ the probability 
that the particle, on starting out from A, is at B after n moves. If the particle 
starts out from point A, the event that the particle is at A after nm — | moves 
and the event that this particle is at B after n — 1 moves form a complete 
system of mutually exclusive events; hence, on the basis of Property 7, 
Pu™ = pur Ypir + pie Ypai. 
On the other hand, pyy*-) + pig") = | (Property 3); from this it follows 
that 
Pu™ = piu Vp + (1 — pir Y)par 

= pur" pry + par — pir per 

= pai + (piu — par)pir». 
If we set 

Pu—- pi=¥yq 

we obtain 


Pu = par + pu) 
= por + Q(par + gp) 
= par + Qpar + Fp”? 
= por + Gpar + 9 pai + 9p) 
= por + Gpar + 9’pai + Gp” 
= por + Gpor + Q?par t+ +++ + 9? par + Qh purl. 


65 


But, 
Publ)! = py! = pi, 
from which we obtain . 
pu = panl+q+¢ + oe + gr?) + pir, 


or, if we sum by the formula for geometric series,} 





1 — grt 
Pu = a ae +f pun = I i + qr (Pus = =a): 
By taking into consideration that pii1 + pi2 = 1 (Property 3), we obtain: 


l1—g=1—pi + poi = Piz + Pai, 


(n) — _ P21 =a ee a) 
Pu Pi2 + Pai pe ie Piz + p21 


epee 2 aerttgy ee oe tie oc ane 2 
P12 + Pai Piz + pai 
ee 4 gr3 Pup — pail — pi) 
Piz + Pei Pi2 + pai 
oe Pah — : n-1 P12P11 — P21pP 12 
pra + par a ae ae Piz + pai 
P21 


“a a 

(6) To find the probability pz1™ that the point A is reached after n moves 
on starting out from B, we first calculate pi2™. We find this probability by 
the formula 


Pe™ = 1 — pu™ 


oy P21 = Piz 22 7 
Piz + po. = Pre + Pai (pu — Pai) 
—_ _ Pr P12 


7 Piz + Pai ~ pie + poi (Pi — Pas)”. 


The probability p21 arises from the probability p12 when we exchange the 
points A and B and, correspondingly, replace the index 1 by 2 and 2 by 1 
Hence, 

P21 


(yy) — —_4** coh nn POE = 7e 
ce P21 + Pi2 Pai + P12 (Pee — pra). 


1See, for example, I. S. Sominskii, The Method of Mathematical Induction (Boston: 
D. C. Heath and Company, 1963), pages 14 and 40. 
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Remark. The probabilities pi: and po, differ by the quantity 


P12 = 54 P21 
Piz +°pPai (Pui Pay” + Piz + pai 


With the exception of the degenerate cases 


Pu an Pa”™ = (pee — P12)". (1) 


Pil = 1, Pi2 = 0, pai = 0, poe = 1, 
Pu = 0, Piz= lL; Pa = 1, P22 = 0, 


we always have that 


—l<pu - pa <1, 
—1< poe — pie <1. 


Hence, as n increases, the summands of the right-hand side of equality 
(1) approach zero, since they are terms of a decreasing geometric progression. 
Hence, the difference p11™ — p21 also becomes ever closer to zero. In other 
words, with increasing n, the probability that the particle is at the point A 
after n moves becomes increasingly independent of the point from which it 
starts out. 


PROBLEM 6. We shall represent the number of balls in the left urn by means 
of a marker that moves on the line of numbers. At the beginning, the marker 
is at point a (Fig. 27). In each unit of time it moves with probability } to the 


Pe 
ANT” 
Co eS ee ee oe ep 
OO 2 Ke a+b—-l a+b 


Fig. 27 


right (if a ball is moved from the right urn to the left, so that the number of 
balls in the left urn increases by one), and with the same probability, to the 
left (if a ball is moved from the left urn to the right, so that the number of 
balls in the left urn decreases by one). This goes on until the marker reaches 
the point 0 or the point (a + 5) for the first time. If the marker reaches the 
point 0, the left urn has become empty, and if it reaches the point a + 5, the 
right urn has become empty. We denote the probability that the marker 
reaches the point 0, under the condition that it starts out from the point k, 
by px (Pa is sought). Obviously, po = 1 and Pas» = 0. 

Suppose that the marker is at point k. 

We consider the two events: 

A: The marker is at k + | after one move. 

B: The marker is at k — | after one move. 

By assumption, P(A) = P(B) = 4. If the event A occurs, the probability 
that the marker reaches the point 0 is equal to px41, and if the event B occurs, 
this probability is equal to px_1. 
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By the formula for complete probability, we have 


Pr= F pass + J Pa 


from which we obtain we 


2Pk = Pr+1 + Pk-1s Pr+1 — Pre = Prk — Pk-1- 


We denote the constant difference p.41 — px by d and write 


Pre — Pri= d, 
Pr-1 — Pr-2 = d, 
p2—-pr1=4, 
Pi —pPpo= d. 
By addition of these equalities we obtain 
Pr — po = kd, 
Pr- l= kd, 
Pr= 1 + kd, 


or, setting k =a + b, 


1 
a+b’ 
k _a+t+b—k 
a+b a+b — 


0 = pary = 1+ (a + Dd, d=— 
pe=1l— 


The probability in which we are interested equals 


5 
a+b 





Pa = 


The probability that the right urn becomes empty is then, obviously, 


a 
a+b 





Po= 


The probability that the experiment ends, that is, that one of the urns becomes 
empty, is, by Property 2, 
b a 
=o) 
a+b a a+b 








The probability that the experiment never ends is, by Property 3, 
1-1=0 


Remark. This problem is known in the history of mathematics as the 
“gambler’s ruin problem.” Its classical formulation is as follows: 

Two people gamble. The probability of a victory for each of them during 
each game is equal to 4. The first gambler has a rubles, the second b rubles. 
Play continues until one of the players loses his last ruble. What is the prob- 
ability of ruin for each of the gamblers? 
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In Problem 6, the players are replaced by urns and the money by balls. 


PROBLEM 7. At each vertex of the cube we write y 0 
the probability (Fig. 28) that the caterpillar after 

leaving this vertex gets stuck at the point A. The , oN a 

same probability x is written at the vertices | and 
5 of Figure 5, since these vertices have the same 
position with respect to the points A and B. Like- 
wise, the same probability y is written at the ver- 
tices 2 and 4. We denote the probability that the 
caterpillar crawls to A from the points 0 and 3 by 
z and u, respectively. 

We now assume that the caterpillar is at the 
vertex 1 (Fig. 5), and consider the complete system of pairwise mutually 
exclusive events: 

C,: The caterpillar crawls along the path 14, 


2 x 
Fig. 28 


C2: The caterpillar crawls along the path 12, 
P(C2) = + 
C3: The caterpillar crawls along the path 10, 


ame 
P(C3) = 3" 

The probability that the caterpillar reaches the point A is equal to | under 
the condition Cj, equal to y under the condition C2, and equal to z under the 
condition C3. Hence, (by Property 7), 

aes ap egal 1 
Seq l+agy+* () 


Similarly, one finds the three relations 


yopOtdxt iu, (2) 

padxt art yl (3) 

u=ty tty tye (4) 
We find 


as solutions of the equations (1), (2), (3), and (4). 


The probability z’ that the point B is reached if the caterpillar leaves 
from the point 0 is found by the following considerations. Obviously, the 
probability that B is reached from the vertex 0 is equal to the probability that 


the point A is reached from the vertex 3. Hence, 2’ =u = 5. By the same 


considerations, we find 
j 5 4 
SPS ay a a aa 


Remark. We remark that 
CX apt pate jap wal: 


But, by Property 2, z + z’ is the probability that the caterpillar reaches 
either point starting from the vertex 0, and 1 — (z + z’)is the probability that 
the caterpillar reaches neither point starting from the same vertex (Property 3). 

Thus, the probability that the caterpillar reaches neither point A nor 
point B on starting out from the point 0 is equal to zero. One easily sees that 
it is equally true when any other vertex is chosen as the starting point. 

We have already met a similar result in the solution of Problem 6, where 
it was shown that the probability that the marker reaches neither 0 nor 
a + b is equal to zero. The probability that the game of Problem 4 never 
ends is also equal to zero. Finally one can easily ascertain that in the diagram 
of Problem 5, the probability that the particle never reaches, say, point B is 
likewise equal to zero (if p12 > 0). 

All these facts are special cases of the general theorem about random 
walks, which we prove in Chapter 2. 


PROBLEM 8. We consider the events: 
A: After n moves, the marker is at point k. 
B,: After n — 1 moves, the marker is at point k — 1. 
Bz: After n — 1 moves, the marker is at the point k + 1. 
B3: After n — 1 moves, the marker is neither at kK — 1 nor atk 4 1. 


By Property 7, 
P(A) = P(B:) P(4|B:) + P(Bz) P(A|B2) + P(Bs) P(4|Bs), 
and we have, 


P(A) = Zy#, P(B1) = Zna¥-3, P(Bo) = Zp_s#4, 
P(A/B:) = 4, P(A|B2) = 4, P(A|Bs) = 0. 
From this, it follows that 


Zn_\*71 Zn kth 
Zea 4 n—-1 . 
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PROBLEM 9. The elements that stand in the rows of the triangle of prob- 
abilities are the probabilities of events that are pairwise mutually exclusive 
and form a complete system. 


PROBLEM 10. For this proof, we write three products, one under the other 
(compare with (5), (6), and (7) on page 22): 









































1,1,3.3....,24=3,2a-3 2a-1 2a-1,_ 2a 
22 4 4 2a—2 2a—2 2a 2a 2a+1 
y 2441... 2kK=2, 2k=-1 
2a+2 2k — 1 2k 
My Jol 9 BP ee , 2a —3 2a—3 2a-—1 d2a-1 _ 2a+41 
22 4 4 2a—2 2a—2 2a 2a 2a +2 
2a + 1 2k—1 2k-—1 
2a+2 2k De. ? 
2 Peet ee See oe _2a—3 2a-3 2a-1, 2a  2a+1 
22 4 4 2a—2 2a—2 2a 2a+1 2a+42 


2a+2.  (%-1. 2% 
2a +3 eo eT 








We see, immediately, that the second product is not smaller than the first, and 
the third is greater than the second. However, the second product is equal to 
Woz”, and, after a few simplifications, the first and third take the form 











ier eet omy aca ee Fe 2a_dat+ld,... 2k=-1 
4 























2 2a —2 2a 2a 2a+l 2a+2 2k 
=(5-g0 ey 4 et 
TAQ 4 2a —2 2a 2k’ 
(i-3 = 3¥ Jad, 2a da+1 2a+2. | 2 
2 4 2a —2 2a 2Za+1 2a+2 2a+3 2k + 1 
=( 40 SS) 
~A2 4 2a —2 2k+1° 
Hence, 
1.3...,,2a—3 2a—-1 1 
Qa-1(>-4 yea) 2a ak 
— 3\2 
< Wo? < (2a — De (Fg HS) re 


and the desired inequality follows, on taking the square root. 
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PROBLEM 11. For k > 150 (inequality (16) on page 24), 


1 1 
<w ——., 
Juisk = "** Satan 
from which it follows that a“ 





l 
7749 Vk ~** Sao VR 


0.5633 0.5644 
Ee <i Wer < a) 


= 5,000, and, hence, 








In our case, k = ee 


70.710 << Vk < 70.711; 0.01414 < SE < 0.01415. 


Finally, we obtain 


0.5633 x 0.01414 < wo, < 0.5644 x 0.01415, 
0.007965 < Wer < 0.007987. 


On rounding off this value to two significant digits, we have 
Wo, = 0.0080. 


PROBLEM 12. Let n = 2k. All terms of the nth row are smaller than the 
middle term, which satisfies the inequality 


] 1 


W, = Wo. << ——= = — _. 
n 2k 2k Vn 
Now, let n = 2k — 1. There is no middle term in odd-numbered rows, and 
the largest terms in these rows are the equal terms Z,,~! and Z,}. But, 








4 1 
Zr = Zn t+ Znt — av ee) 
2 
As just shown, 
1 
Znt1? ‘ 
eS ioe 
Hence, 
Ie 2. t 
Zy) —; 
saa ya ba Vn’ 
the other terms of the nth row are thus certainly smaller than a 


n 
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PROBLEM 13. Since the sth term from the middle term of the 2kth row 
is Z2x?8, one can rewrite the inequality (19) on page 26 in the following way: 


k+1— i Zon? ( k ) 
Sh SS < 
( k+1 ~ Wor — \k +s)? 








or, 





For k > 60 (see inequality (15) on page 24), 


: < Wor << h=3 


3.18k V3.10k 








Hence, for k > 60, 
1 (- +l-—s 
V 3.18k k+1 


For an approximation to Z;29?°, one must set s = 10 and k = 60. Then, 


Lo (34) < Zino < —L_ ()", 


\/3.18-60 \61 \/3.1-60 \70 





8 k s 
) < Lox?8 < (—} 2 
3.10kK \‘k +s 


and, hence, 


0.012 < Zy29?° < 0.016. 


PROBLEM 14. We have 


(1 + py = (1 + pl + p) --- (1 +). 

r times 
Multiplying out these factors, we obtain a number of terms. One of the terms 
is 1, obtained from multiplying together the 1 from each factor. After that, 
we obtain p by multiplying p from the first factor by | from all the other fac- 
tors; we likewise obtain p on taking p from the second factor and multiplying 
by 1 from all the other parentheses, etc. Hence, p occurs r times in the 
expansion and 


Ch py Pp hp kp) a ee 


rterms 


l+mpte--. 
The terms we have not written out are all positive; hence, 


(l+p)’>1+ 7. 
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PROBLEM 15. One can ake ey \/n = 5.429 \/n as the practically certain 


W/0.05 
bound for the displacement after n moves. In particular, for n = 1,000, 171.7 
is the practically certain bound of the displacement. The largest even number 
that does not exceed 171.7 is 170; hente, one can state with practical certainty 
that the marker is not more than 170 steps from the starting position after 
1,000 moves. 

To obtain the practically certain bound for the reduced velocity, one 
must divide the bound for the deviation by the number of moves. We find 
that, with practical certainty, the marker has a reduced velocity of at most 
0.17 after 1,000 moves. 


PROBLEM 16. We set 





e= loa): 


One can say with the probability of error less than e that after n moves the 
2 
. But, 


Yevn 


reduced velocity of the marker is less than 





SDE oy) OD oy 
Vevn 2 
a 


PROBLEM 17. One can assert with a probability of error less than 0.001 that 
2 


0.001 \/n 


after n moves the reduced velocity of the marker is less than 


We now choose n in such a way that 


2 
+S — < 0.01; 
0.001 \/n — 
from this, it follows that n > 4- 108. 
Hence, one can assert with the probability of error less than 0.001 that 
the reduced velocity after 4-106 moves is smaller than 


2 = 0.01, 


W/0.001 \/4- 108 


PROBLEM 18. In order to assert with probability of error less than 0.01 that 
the frequency with which heads comies up differs from 0.5 by less than 0.1, 
we can take any number of tosses that is greater than 


1 = 464.17. 


"= op 0.01 


It is thus sufficient to make 465 tosses. 
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PROBLEM 19. If the particle is at m after n moves, it could have arrived there 
first after one move, after two moves, ..., or after n moves. Of interest to 
us is the probability that is expressed, with the help of Formula (5) of sec- 
tion 2, as the sum 7 


by + bo +--+ +b, 


where 5; is the probability that m is reached for the first time after k moves. 
Hence, it suffices to show that the probability b; does not depend on a barrier. 

We now prove that b; does not depend on what kind of barrier is at the 
point m. We consider all paths of k moves that begin at the starting point, 
end at the point m, but otherwise do not touch the point m. These paths are 
denoted by Aj, Ao, ..., As. We denote by a the probability that the particle 
uses the path A; for its first k moves. The event that the particle reaches the 
point m for the first time after k moves is the same as the event that it used 
either the path A;, the path Ag, ..., or, finally, the path A;. Thus, 


by = G1 + dg 4+ --> + 4s. 


However, none of the paths Aj, Ao,..., As goes through m; hence, the 
probabilities a1, a2,..., a; (and with them also b,) do not depend on 
whether or not there is a barrier at the point m or what kind of barrier it 
may be. 

Remark. These considerations have a general character and can be applied 
to every scheme of a random walk. Hence, the statement formulated in the 
exercise is valid for arbitrary schemes. 


PROBLEM 20. From the relations (1), (2), and (3) on page 38, we find that 


But, bo = | (at the beginning the particle is at 1) and b, = 0 (after one move, 
the particle is no longer at 1). Hence, 


bo = (3)° (1) 


bor+1 =0: 


It follows from the equalities (3) and (4) on page 38 that 


dy, = G1 + (4) by_2. 


ee) 


In addition, by taking into consideration equation (1), we obtain 
Gon41 = da + qbok-1 = d2zy 


M4 
dex, = doxr-1 + jp bak-2 + dox-2 + — (3) 


4 
5 I (3\F2 I {3\} 
= dos + 7 (3) + + (3) 
= 1/3), 1/34 ...,1 3)" 
= +7(4) +4(a) + +3 (4) 


Since do = 0, we finally obtain (by using the formula for the sum of a geo- 
metric series) 


(6) 
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PROBLEM 22. (a) The second particle, just as the first, can also reach E; from 
£, in z moves.! If we relate this path of the two particles to the motion of the 
marker on K?, we find that the marker reaches Ey, from E,, in z moves. 

(b) It suffices to prove that one can reach E,; from an arbitrary position 
Ey, of the chain L. (One can, then, also reach any arbitrary position Ey 
from it along the path E;, — Ey, — Ej.) Thus, we have to show that one can 
reach £1; from the position E;, belonging to L. Suppose that one reaches the 
point £;, from £1; after s moves. In this case, the first particle goes from £, 
to E; in s moves, and the second goes from E to E, (Fig. 32). Now, let Ey 


Ss Ss 
Be ES 
fo) oF; 
Pa ee been 
r t 
Fig. 32 

be reached from £; in r moves and £, from £;, in t moves (the chain K is ir- 
reducible!). Then (Fig. 32), the first particle can go from £; to Ejinr +s +¢ 
moves (on the path E; — E, — E, — £)) and the second particle from EF; to 
Ey int +s +r moves (on the path E;, — £, — £; — E,); that is, the first par- 
ticle requires the same number of moves to go from £; to £, as the second 
does to go from £; to £,, namely z = s +r+t moves. If this path of the 
two particles is carried over to the motion of the marker on K?, the marker 
can reach £), from £;, in z moves. 
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PROBLEM 23. Since all points are fully equivalent, x is really the probability 
that a particle leaving n — | reaches n at some time. Likewise, y is the prob- 
ability that a particle leaving n + 1 reaches n. 

We assume that the particle starts out from the point 0 and consider the 
events: 

A: The particle returns to the point 0 at some time. 

B,: The particle reaches the point | on the first move. 

Bz: The particle reaches the point — 1 on the first move. 

By the formula for complete probability (Property 7, section 3), we have 


P(A) = P(Bi) P(A|Bi) + P(B2) P(A|Bo). (1) 


If the condition B, is fulfilled, the particle is at the point 1 after one move. 
For the event A to occur under this condition it is necessary that the particle 
reach 0 at some time on starting out from 1. The probability for this is equal 
to y. Hence, P(A|Bi) = y. Likewise, P(A|B2) = x. Furthermore, P(A) = z 
and P(B,) = P(Bz) = 4. If these values are substituted in formula (1), 
we obtain 


sillpecge sls 2 
Z=aV+ 5% (2) 


Furthermore, it is obvious that x = y, and, thus, z = x. 


1“Can reach” means “can reach by moving in the direction of the arrows.” 


To calculate x, we consider the event C (that the particle reaches | from 
0 at some time). By Property 7, we have 
x = P(C) = P(B,)P(C|Bi) + P(B2) P(C|B2). (3) 


We have P(B,) = P(B2) = $ and P(C|R,) = 1. Furthermore, P(C|Bz2) is the 
probability that the particle reaches 1 at Some time on starting from —1. We 
shall calculate this probability. Thus, suppose the particle is located at —1. 
The probability of the event D (the particle reaches the point 1 at some time) 
must now be found. For this, we introduce the event F (the particle reaches 
0 at some time). Obviously, FD = D (for, the occurrence of D is equivalent 
to the joint occurrence of the events F and D). Hence, by Property 6, 


P(D) = P(FD) = P(F)P(D|F). 
But, PF) = x, P(D|F) = x. Hence, 
PUD) =x": (4) 
Finally, we find from the formulas (3) and (4) that: 


RS Gs (5) 
x?—2x+1=0, 
(x — 17 =0, 


from which it follows that x = 1, and, thus, y =z = 1. 


PROBLEM 24. It must be shown that the particle reaches the point ” with 
probability 1 (for the sake of definiteness, let n > 0). For n = 1, this was 
proved in the previous exercise. Let the statement be proved for the point n; 
we will prove that it then holds for n + 1 as well. 

Let us consider the event An, (the particle reaches n + | at some time). 
Obviously, Anyi = AnAnsy1; by Property 6, we then have 


P(An+1) = P(Az) P(An41|An). 


But, P(4n41/4,) = x = 1. Furthermore, P(A,) = 1, by assumption. Hence, 
P(Ans1) = 1. 

The probability that a preassigned point is reached twice, three times, or, 
in general, n times, is found exactly as in Remark | on page 43; it is equal to 1. 


PROBLEM 25. We remark first that p + p + r = 1. (See footnote | on page 46.) 
We may denote by x, y, z the probabilities of the same events as in Problem 
23. We leave it to the reader to find for himself the equations, 


BEE Pe AP, (1) 
X=ptrx t+ px. (2) 
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(One derives them exactly as (2) and (5) in the solution of Problem 23. One 
has merely to consider, in addition to B, and Bo, the event B3: the particle 
remains at the point 0 after the first move). Obviously, x = y. Hence, 


zZ=r + 2px. 
We transform (2): 
x=p+(l — 2p)x + px’, 
O = p — 2px + px?. 
Since p £0, 
1— 2x 4+ x? = 0, 


from which one obtains x = 1 andz =r-+ 2p = 1. 
Exactly as in Problem 24, we ascertain that the particle reaches every 
point with the probability 1. 


PROBLEM 26. With the same considerations as were applied in the solution 
of Problems 23 and 25, we obtain 


2 = py + 4X, (1) 
x =p + qx, (2) 
y=q+ py (3) 
where, in general, x  y. The equation (2) yields 
1+ V1 — 4pq 
= = ogee 
_ 1+ vil — 4p(l — p) 
= 24 
Le vd— apy 
= ay meee 
_ 14 — 2p) 
= a” a 
as solutions; the equation (2) thus has the roots 
l+1l1—2p 2-2 I1-p q_4 
SS SS eS ee Ee ; 
24q 24 q q 
_1l=-l+2p_ 2p _?P 
eae?” a a 


Since ee 1 and x (like every probability) is less than or equal to 1, x # x2 
and, hence, x = x; = 1. 


79 


We obtain 


y= 1, yad 


\ 
as a solution of equation (3). By assumption, y # 1, and, therefore, y = o 


Hence, 


z= py + qx = 29. 


PROBLEM 27. We prove that for k > 2™ the inequality S; ya is satisfied. 











We have 
1 ] Pe 
Some Ne ah 2 de Rose sda 
where 
Ae te ee ig ie 
=p 1 toaqat te 
2?-1 summands 
Clearly, 
l 1 Le eee 
Ip 25g opt $5. 5: 
2?-1 summands 
Hence, 
“ a get diss az.e Fae ete m 
Sk 2 So eta hob 5 eee er 





m summands 


PROBLEM 28. We consider the straight lines NyNi, N_1N_1, N2N2, N_2N_2, 
that run parallel to the line NN (Fig. 26). Let a traveler be at a point of the 
line N;,N;,. The traveler goes upward on the next move with a probability 4 
and downward on the next move with a probability 4; that is, he remains on 
the line N,N; with the probability 4 + 4 = 4. The traveler goes to the right 
and reaches the line Nx,1Nx41 with a probability 4 or goes to the left and 
reaches the line N;_;Nx_1 with the probability +. If we regard the line N,Nx, 
as one position, we obtain the situation that we have already dealt with 
in Problem 25. The lines N,N; here correspond to the points, with p = 4 and 
r = 4. By what was proved there, the traveler reaches each line with proba- 
bility 1; thus, also the line NN. 
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Ranpom Wa ks is a translation of Part Three of Mathematical 
Conversations by E. B. Dynkin and V. A. Uspenskii, which 
was published in the Russian series, Library of the Mathe- 
matics Circle. The originality of the exposition and the variety 
of the problems presented here make this booklet especially 
useful in stimulating an inventive approach to mathematics. 
Extensive solutions to all problems are provided. 


This booklet deals with some of the more elementary problems 
in probability theory. The exposition ranges from the simplest 
examples of a random walk on a line to such more complex 
examples as random walks through a city, and Markov chains. 


The booklet is designed for the reader’s active participation, as 
the problems are carefully integrated with the text and should 
be solved in sequence. The reader should have a background 
of high school algebra. 

E. B. DYNKIN, a Professor at Moscow State University, is an 
eminent mathematician and author, whose specialties are 
higher algebra, topology, and probability theory. V. A. 
USPENSKII, a Lecturer at Moscow State University, spe- 
cializes in mathematical logic. 


